When does the second phase open?

Hello all,

sorry for all the trouble with the transition from Phase 1 to Phase 2 - @dipam just fixed the problem with the sorting - so this should be fine now.

@subatomicseer Is there still a problem with missing submissions for leaderboard A?

We are also working on adding the old submissions from Phase 1 to the leaderboard of Phase 2 - this will allow you to see the effect of overfitting of your models. If everything goes well, this should be done by end of the week.

Kind regards

Stefan

View post 2

When does the second phase open?

Hello @JusperLee and @subatomicseer ,

sorry for the delay - I have pinged AIcrowd and hope that they will look into it as soon as possible. It is only a visualization problem and none of your submissions will be lost.

Kind regards

Stefan

View post 2

MDX: UMX Baseline for Leaderboard A and Leaderboard B

Hello all, here are the first baselines for Leaderboard A and B of the MDX track:

• UMX trained on label-swap dataset: Open-Unmix-Pytorch LabelNoise | Zenodo
• UMX trained on bleeding dataset: Open-Unmix Pytorch Bleeding | Zenodo

These models are trained directly on the noisy datasets without paying special attention to obtain robust models (which is something that you should definitely explore ).

You can already find their scores on the corresponding leaderboards AIcrowd | Music Demixing Track - MDX'23 | Leaderboards and AIcrowd | Music Demixing Track - MDX'23 | Leaderboards.

Kind regards

Stefan

View post 3

What baselines are coming up?

AIcrowd will do this (and also upload the video to YouTube), right @snehananavati ?

View post 1

What baselines are coming up?

Hello @alina_porechina, during our town hall on Saturday, we announced the following upcoming baselines for the MDX track:

image1916×1085 261 KB

For the CDX track, there is now one new model from MERL that was trained with the SDR loss - please see here: GitHub - merlresearch/cocktail-fork-separation: Baseline multi-resolution cross network model trained using the Divide and Remaster Dataset

Kind regards

Stefan

View post 1

MDX: failed submissions count towards the daily limit

I think there are two different counters for successful and unsuccessful submission - but I am not sure.

@dipam Do you distinguish between successful and unsuccessful submissions?

View post 1

MDX: failed submissions count towards the daily limit

Hello @alina_porechina,

I think you hit the maximum number of submissions/day. If you look at AIcrowd | Music Demixing Track - MDX'23 | Submissions then you can see in the upper right corner how many submissions you have left for this day:

Kind regards

Stefan

View post 1

Evaluation failed

Hi @JusperLee,

maybe this is related to the warning that you get about the debug mode? I am not sure where to enable the debug mode but maybe he only does a partial evaluation and then stops?!

Did you check the log files (they should appear at the end of the submission issue).

Another reason I could think of would be a timeout, i.e., if the evaluation took too long.

Kind regards

Stefan

View post 0

MDX track: UMX-L and X-UMXL baselines available

Hello all,

quick update: we renamed XUMX-L to XUMX-M in the starter-kit in order to avoid any confusion with UMX-L and XUMX-L as they were trained on different datasets.

M stands for medium as XUMX-M was trained on ~100 hours of music compared to the ~400 hours of UMX-L.

Kind regards

Stefan

View post 1

MDX track: UMX-L and X-UMXL baselines available

Hi @subatomicseer

I just wanted to know about the scale of external data used for UMX-L. It just says private compressed stems, but could you tell us how much extra data it was trained on?

I also don’t know the details - only what @faroit wrote here: https://twitter.com/faroit/status/1413409137032060929

@faroit Do you remember how many hours the dataset corresponded to?

Kind regards

Stefan

View post 0

MDX track: UMX-L and X-UMXL baselines available

Hello all,

just to let you know that you can now find two baselines in the MDX starter-kit

• UMX-L in umxl_music_separation_model.py - more details here.
• XUMX-L in xumxl_music_separation_model.py - more details here.

Please note that both models can not be used for Leaderboard A and B as they used other data for training.

Kind regards

Stefan

View post 1

Cocktail-fork model now available in starter-kit

Hello everyone,

the starter-kit for the CDX track just got updated and now contains also the cocktail-fork model from MERL.

Checkout here: cocktail_fork_separation_model.py

Kind regards

Stefan

View post 1

Structure of the competition

Hello @denis_mekhanikov, sorry for my late reply - I just saw your message now.

Regarding your first question: This should be fixed now and you can see three leaderboards here: AIcrowd | Music Demixing Track - MDX'23 | Leaderboards. Please make sure to tag your submission as described in Leaderboard A (label-noise) and Leaderboard B (bleeding). All submissions will automatically be listed in Leaderboard C. For Leaderboard A and B, you need to use the datasets that are provided here. For Leaderboard C, you can use any training data.

Regarding your second question: Please note that we updated the starter-kit with the start of the challenge last week. We used the old starter-kit for the warm-up round (hence, there was also one leaderboard as you noticed).

Please let us know if this answers your question

Kind regards

Stefan

View post 0

Aicrowd-bot adds people to my private submission repository for CDX track

Hi @subatomicseer , no - this is a bug which needs to be fixed. We should not get access to your repositories as these are private.

View post 2

Dataset explanation

Hi @sandesh_bharadwaj97, you can preprocess the datasets and try to employ an automatic method that can reduce the impact for the DNN training, e.g., for the bleeding you could look into methods like

Prätzlich, Thomas, et al. “Kernel additive modeling for interference reduction in multi-channel music recordings.” 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015.

We have created the datasets such that it should not be possible to just “clean them up” by manually labeling or doing a simple signal processing based filtering:

• For the label swap, the label swap happens before mixing to four stems: E.g., if a song contains the five instruments “bass guitar”, “drumset”, “male singer”, “male background choir”, “synthesizer”) then we, e.g., put “male background choir” into “other” and mixed everything to the four stems “bass”, “drums”, “vocals” and “other” - now “other” contains a mixture of “vocals” and “other” and, hence, is a noisy target for training.
• For the bleeding, we used a set of random effects (reverb, gain, shifts) to create the bleeding and a simple filtering/summation should not be sufficient.

With these two datasets, we want to investigate how good models can become if they are trained on noisy training data (e.g., by modifying loss function, regularize the networks, …).

Please note that you can not use any model which is trained on other datasets than the one that you are allowed to use for leaderboard A or leaderboard B, respectively.

Kind regards
Stefan

View post 2

Dataset explanation

Hello @subatomicseer,

Will the stems still sum to the original mixture? What I mean is for example, if drums = drums + 0.2*bass, then will the bass be reduced to 0.8*bass or it will still be the original bass?

No, this is not the case - the bleeding comes on top and, hence, it is an additional component for every stem.

Please note that the two new datasets do not contain a “mixture.wav” but only the stems “bass.wav”, “drums.wav”, “other.wav” and “vocals.wav”. From this you could create the mixture yourself by simply summing the four.

Will the test GT stems also have bleeding or they will be clean?

No, they do not have bleeding. Actually we use the same hidden evaluation dataset for all three leaderboards (MDXDB21 which is described here) such that results are also comparable across leaderboards. It will then be interesting to see how much models suffer from having to learn from imperfect data (containing label swaps or bleeding).

Kind regards

Stefan

View post 0

Dataset explanation

Hello @sandesh_bharadwaj97, welcome to SDX 2023

for the corrupted datasets i.e Leaderboard A and B, its mentioned that we can only train on those datasets. Does that mean we cannot use pre-trained models and fine-tune them on the corrupted datasets?

Yes, for MDX Leaderboard A and Leaderboard B, you can only use models that are trained on the provided datasets and you are not allowed to finetune models that were pretrained on other datasets (even if they were trained on datasets like ImageNet from Computer Vision which is another domain than audio).

View post 1

Dataset explanation

Hello @heaven, yes - we will provide specific datasets for leaderboard A and B in Track A which participants need to train on. They contain wrongly labeled stems (leaderboard A) or bleeding between stems (leaderboard B). We are currently preparing them and will publish them when the challenge starts (currently, there is only a warm-up round which allows participants to experiment)

View post 0

Dataset explanation

Hello @heaven,

this dataset is meant as a preview version of the original MUSDB18 dataset (much smaller in size as there are only 7 seconds of the original audio) and was created with this and this code. It contains 7s from the original songs where there is most activity and which can thus be used as a preview of the original dataset.

View post 0

Are we allowed to use a pretrained model like openl3

Hello @joseph_turian, sorry for my late reply - I missed your message.

I just answered a related question here. Does this answer your question?

View post 0