Librimix recipes #418

JorisCos · 2021-01-25T21:09:08Z

About this PR

This PR adds new recipes to the librimix dataset :

DCCRNet
DCUNet
DPRNNTasNet
DPTNet
SuDORMRFNet
SuDORMRFImprovedNet

All the models are trained for speech enhancement on librimix 16khz train-360.
The pretrained models have been uploaded to Hugging Face (@jonashaag can you take a look at DCUNet the performance are quite poor does it looks similar to your experiments ? )

Beside the new recipes I changed the data generation of this recipe. The local data generation i.e the csv files with the path to the audio is separated from the LIbriMix generation (it should have been in the first place).

I also introduce a new function declipp for declipping audio before feeding it to the ASR models.

There are a lot of duplicates in the code as I can't make the folder local a symbolic link. This probably falls under the discussion of the recipes' design.

I realize that it's a lot of files to check... all recipes have been tested and should work though.

Coming soon

Soon i will upload the pretrained models and results for SuDORMRFNet, SuDORMRFImprovedNet
I am also thinking about extending the evaluation with WER computation.

jonashaag · 2021-01-25T21:41:56Z

😍

mpariente · 2021-01-26T10:59:40Z

Oh yes, lots of review to do 😂
Could you highlight the points of attention you'd like in the review please?

jonashaag · 2021-01-26T13:41:06Z

Concering the DCU performance, could it be because of different context (only 2 s)? Can you show training loss graphs of each of the models?

Btw I'm surprised by ConvTasNet and DPRNN performance, it was much much worse for me (but I did dereverberation, not denoising)

JorisCos · 2021-01-27T13:22:02Z

Concering the DCU performance, could it be because of different context (only 2 s)? Can you show training loss graphs of each of the models?

Btw I'm surprised by ConvTasNet and DPRNN performance, it was much much worse for me (but I did dereverberation, not denoising)

It might be because of the shorter context I can increase it if I reduce the batch size. I will send you the graphs.

Oh yes, lots of review to do 😂
Could you highlight the points of attention you'd like in the review please?

Can you check one run.sh see the changes in the data generation. Also, the declipp function and one eval script. Otherwise it's pretty much the same as the first recipe

mpariente

Overall, it looks good, thanks a lot.
Few comments to address that I highlighted in the code.

The duplicate on local is a disaster, but that's bad design of Asteroid recipes indeed.

mpariente · 2021-01-27T19:23:35Z

asteroid/dsp/declipp.py

+    for i in range(len(est_np)):
+        est_np[i] *= np.max(np.abs(mix_np)) / np.max(np.abs(est_np[i]))
+    return est_np


This modifies the array in-place, which is a bad idea.

I'd use list comp here

mix_max = np.max(np.abs(mix_np) return np.stack([est * mix_max / np.max(np.abs(est)) for est in est_np], dim=0)

This needs a test.

mpariente · 2021-01-27T19:26:37Z

egs/librimix/ConvTasNet/README.md

 | train-360 | sep_noisy |    12     |  12.5   |
+
+See available models [here](https://huggingface.co/JorisCos).


There are filters on the Hub:

You can see all Asteroid models here: https://huggingface.co/models?filter=asteroid

All Asteroid on Libri1Mix here: https://huggingface.co/models?filter=asteroid,dataset:Libri1Mix

So two things:

I think we should have a LibriMix tag instead of Libri1Mix so that all LibriMix models are accessible from one query.

For now, let's link to here: https://huggingface.co/models?filter=asteroid

mpariente · 2021-01-27T19:36:11Z

asteroid/dsp/declipp.py

+import numpy as np
+
+
+def declipp(mix_np, est_np):


I'm unsure about the name. Declipping is the task of restoring the signal after clipping.
Maybe normalize_estimates(estimates, mixture) would make sense? The name is less pretty but more explicit.

@jonashaag

I will change the name but should I keep est_np as variable name to indicate that we only support np.array for now ?

Yes, keep the est_np but if you can change the order of the variables it would be cool.

mpariente · 2021-01-27T19:36:35Z

asteroid/dsp/declipp.py

+    """
+
+    Args:
+        mix_np (numpy array): One mixture


For typing to work, you need to use np.array.

mpariente · 2021-01-27T19:36:58Z

asteroid/dsp/declipp.py

+
+
+def declipp(mix_np, est_np):
+    """


Need a docstring explaining simply what the function does.

egs/librimix/ConvTasNet/local/prepare_data.sh

mpariente · 2021-01-27T19:40:19Z

egs/librimix/DCCRNet/README.md

@@ -0,0 +1,20 @@
+### Results 
+
+The model was train for the `enh_single` task on Libri1Mix `train_360`.


Suggested change

The model was train for the `enh_single` task on Libri1Mix `train_360`.

The model was trained on the `enh_single` task on Libri1Mix `train_360`, at 16kHz.

Same for all I guess.

egs/librimix/DCCRNet/README.md

egs/librimix/DCCRNet/eval.py

mpariente · 2021-01-27T19:45:09Z

egs/librimix/DCCRNet/local/conf.yml

+optim:
+  optimizer: adam
+  lr: 0.001
+  weight_decay: !!float 1e-5


Missing new line

add normalization.py add new lines conf Fix readme black train

mpariente · 2021-02-02T10:39:17Z

Fix linter please

asteroid/dsp/normalization.py

mpariente · 2021-02-02T11:20:58Z

... When there is a commit on the browser, the tests are not ran 😑

There was again a linter problem, did you run black?
Otherwise, it looks perfect.

mpariente · 2021-02-02T11:21:19Z

I spoke to fast, the tests started ^^

JorisCos · 2021-02-02T13:53:46Z

I forgot the test for normalization I will request your review when the recipe is completed

add test normalization_test.py

…recipes # Conflicts: # asteroid/dsp/normalization.py

asteroid/dsp/normalization.py

mpariente · 2021-02-02T17:44:29Z

Thanks again !

JorisCos added 5 commits January 21, 2021 17:26

separate local metadata generation from dataset generation

00def81

add recipes

6286bfc

add recipes

589a8e1

replace utils with sl

4494f19

add sftft_n_filters to config

383fe45

fix epoch and batch size

010ac4a

mpariente reviewed Jan 27, 2021

View reviewed changes

JorisCos added 4 commits January 28, 2021 14:38

hyper parameters for 16 Khz

bc5cd46

remove wrong scheduler from SudoRMRF

9ef7870

same for SudoRMRFImproved

22590d7

remove declipp.py

1cf8717

add normalization.py add new lines conf Fix readme black train

mpariente reviewed Feb 2, 2021

View reviewed changes

asteroid/dsp/normalization.py Outdated Show resolved Hide resolved

Update asteroid/dsp/normalization.py

5a1bf02

JorisCos added 2 commits February 2, 2021 15:20

black normalization.py

cf6d737

add test normalization_test.py

Merge remote-tracking branch 'origin/librimix_recipes' into librimix_…

b4179a6

…recipes # Conflicts: # asteroid/dsp/normalization.py

JorisCos requested a review from mpariente February 2, 2021 14:39

mpariente approved these changes Feb 2, 2021

View reviewed changes

asteroid/dsp/normalization.py Outdated Show resolved Hide resolved

asteroid/dsp/normalization.py Outdated Show resolved Hide resolved

mpariente added 2 commits February 2, 2021 18:30

Update asteroid/dsp/normalization.py

8a0f7a6

Update asteroid/dsp/normalization.py

9aa4ae2

mpariente merged commit 6bed537 into asteroid-team:master Feb 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Librimix recipes #418

Librimix recipes #418

JorisCos commented Jan 25, 2021

jonashaag commented Jan 25, 2021

mpariente commented Jan 26, 2021

jonashaag commented Jan 26, 2021

JorisCos commented Jan 27, 2021

mpariente left a comment

mpariente Jan 27, 2021

mpariente Jan 27, 2021

mpariente Jan 27, 2021

JorisCos Jan 29, 2021

mpariente Jan 29, 2021

mpariente Jan 27, 2021

mpariente Jan 27, 2021

mpariente Jan 27, 2021

mpariente Jan 27, 2021

mpariente Jan 27, 2021

mpariente commented Feb 2, 2021

mpariente commented Feb 2, 2021

mpariente commented Feb 2, 2021

JorisCos commented Feb 2, 2021

mpariente commented Feb 2, 2021

		\| train-360 \| sep_noisy \| 12 \| 12.5 \|

		See available models [here](https://huggingface.co/JorisCos).

		@@ -0,0 +1,20 @@
		### Results

		The model was train for the `enh_single` task on Libri1Mix `train_360`.

	The model was train for the `enh_single` task on Libri1Mix `train_360`.
	The model was trained on the `enh_single` task on Libri1Mix `train_360`, at 16kHz.

Librimix recipes #418

Librimix recipes #418

Conversation

JorisCos commented Jan 25, 2021

About this PR

Coming soon

jonashaag commented Jan 25, 2021

mpariente commented Jan 26, 2021

jonashaag commented Jan 26, 2021

JorisCos commented Jan 27, 2021

mpariente left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpariente commented Feb 2, 2021

mpariente commented Feb 2, 2021

mpariente commented Feb 2, 2021

JorisCos commented Feb 2, 2021

mpariente commented Feb 2, 2021