Training recipes for thorsten dataset

[noranraskin]

Basically just updated the recipes from https://github.com/coqui-ai/TTS-recipes to the new coqpit version of writing training scripts.

[CLAassistant]

All committers have signed the CLA.

[thorstenMueller]

Hi @noranraskin,
thanks for this PR. I'm not sure if @erogol accepts PRs directly to main instead of dev, but we'll see.

Just as side note:
My new neutral dataset (with more natural speech flow) is growing and could be released on early 2022.

[erogol]

thanks for the ✨PR✨ @noranraskin

[erogol]

How about using the new dataset downloader instead of the bash script?

TTS/TTS/utils/downloaders.py

Line 90 in 3780346

def download_thorsten_de(path: str):

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. You might also look our discussion channels.

[thorstenMueller]

Hi @noranraskin, what do you think about @erogol s suggestion?

[erogol]

@noranraskin are there more recipes 😄 on the way or should I just merge away?

[noranraskin]

@erogol there's more recipes on the way. But I'm currently getting an error additional parameter 'ignored_speakers:[] given I also get this on the previously tacotron2-DDC script and all other model recipes, vocoders work fine.
I pulled the latest changes from main a few days ago, any ideas?

[erogol]

Can you try the latest dev? (sorry for the delayed response)

[noranraskin]

@erogol **kwargs was missing in the thorsten formatter. I fixed that issue. Now I'm getting

Traceback (most recent call last):
  File "train_tacotron_ddc.py", line 71, in <module>
    train_samples, eval_samples = load_tts_samples(dataset_config, eval_split=True)
  File "/data/nra/tts-finetuning/TTS/TTS/tts/datasets/__init__.py", line 112, in load_tts_samples
    meta_data_train = [{**item, **{"language": language}} for item in meta_data_train]
  File "/data/nra/tts-finetuning/TTS/TTS/tts/datasets/__init__.py", line 112, in <listcomp>
    meta_data_train = [{**item, **{"language": language}} for item in meta_data_train]
TypeError: 'list' object is not a mapping

On latest main and dev. I'm not getting this error when I try LJspeech. I think this is because some problem with the formatter or data importer, as this dataset only has a single speaker.

[erogol]

New formatters return dictionaries not lists.

[a-froghyar]

Hey, just chipping in with a question here, these recipes won't work without a german text cleaner implemented first, no?

[erogol]

I don't think we need this in 🐸TTS, as you can simply move it to somewhere else in your system.

[noranraskin]

Oh, sorry my mistake

[erogol]

@noranraskin looks good to me. I can merge it when you are ready :)

[noranraskin]

I was still running in some torch errors. I'm updating and testing all the recipes again to make sure everything is working. Might also add a couple more...

[noranraskin]

@erogol so I only got tacotron2_DDC to work.
GlowTTS throws a really long stacktrace with some nvrtc compilation of "default_program" failing.
And all the vocoders through the same cuFFT error at epoch 0.
This could have something to do with my installation, since I'm running a nightly build of torch, as I couldn't downgrade cuda on my system.
All the recipes are copied from ljspeech so I'm running into the same issues there too. Maybe somebody else can check them and report back (haven't opened an issue yet because of the nature of my installation)

Plus:
I added a dataset check at the beginning of all the recipes, that downloads the dataset if it can't be found

[loganhart02]

@erogol so I only got tacotron2_DDC to work. GlowTTS throws a really long stacktrace with some nvrtc compilation of "default_program" failing. And all the vocoders through the same cuFFT error at epoch 0. This could have something to do with my installation, since I'm running a nightly build of torch, as I couldn't downgrade cuda on my system. All the recipes are copied from ljspeech so I'm running into the same issues there too. Maybe somebody else can check them and report back (haven't opened an issue yet because of the nature of my installation)

Plus: I added a dataset check at the beginning of all the recipes, that downloads the dataset if it can't be found

I was able to train glow-tts and hifigan with the recipes past epoch 0. what cuda are you using and have you tried running it on a conda environment with a different cuda?

[noranraskin]

I was running CUDA 11.6 with the compatible pytorch nightly build. But I realised, I was always fetching from 'main' so working with pretty old code. I fixed my installation and tested everything again and now it's working for me too

[noranraskin]

The new recipes for align_tts, vits, wavegrad and wavernn are tested and working.
I'm still getting a RuntimeError for speedy_speech:
Calculated padded input size per channel: (7). Kernel size: (13). Kernel size can't be greater than actual input size.

If you don't know a fix right away I'd just discard this one for now.
Also I'd be ready to merge now, don't have anything to add for anymore.

[thorstenMueller]

Hi @noranraskin ,
first of all thanks for your efforts to add training recipes for my Thorsten dataset 👏.

When the new Thorsten models trained by @domcross and me are released (soon to happen) our next step is the preparation of the dataset release which is the base of the new models. Maybe we can add the recipes to your existing structure then 😊.

Here's a comparison of a model based on the current and the new dataset.
https://www.thorsten-voice.de/2022/03/20/vergleich-thorsten-aktuell-mit-dem-neuen-modell/

[loganhart02]

The new recipes for align_tts, vits, wavegrad and wavernn are tested and working. I'm still getting a RuntimeError for speedy_speech: Calculated padded input size per channel: (7). Kernel size: (13). Kernel size can't be greater than actual input size.

If you don't know a fix right away I'd just discard this one for now. Also I'd be ready to merge now, don't have anything to add for anymore.

Ill see if I can fix the speedy_speech error real quick and then merge :)