My brothers TTS software

[jeroennicolai]

Hi there,

First off thank you for sharing this project! <3
When I first heard of Vall-E my thoughts immediately went to my brother and people that are in similar situations like him.
Seven years ago he was in a horrific motorcycle accident beyond his control and went straight into a coma for 8 months due to severe brain damage, and yes he was wearing a helmet.

After wakening up he has lost almost all of his mobility and ability of speech, he's now bound to an electric wheel and communicates through a TTS app on his iPad. But most if not all TTS applications use the default voices that are built into IOS.
Which keeps on sounding like a stranger with a robotic twist to it. And we miss his voice so much.

In my mind and how I understand the Vall-E paper is that we could train Vall-E with audio from his voice (taken from home video's before his accident for example) and make the new output sound like him or at least get closer to his original voice then we have now.

I would love to go for it and try my best to make this work for him and us, but I'm not an expert in this field.
And would like to ask for some guidance to see if we can make it happen for him and maybe use that as a blueprint for others like him. And with this then show what amazing value this tech has and put some leverage against those stories that are only focussed on the potential real-world negatives of this tech.

Please reach out @lifeiteng and others if you would like to help me, help my brother to regain his voice.
I've attached a quick braindump on how I see the big high over flow just to simplify my thoughts.

Hope to hear from any of you.

J, David's brother

[lifeiteng]

@jeroennicolai

Sorry to hear that.

VALL-E is an auto-regressive model that is quite heavy, and I am trying to improve the model to make it faster and more powerful. In special scenarios, the TTS model should run on the device(I think iPad is enough). I will tell you good news soon.

You can try https://play.ht/ now.

Best,
Feiteng

[jeroennicolai]

Hi Feiteng,

Thank you for your response :), is there anything I could assist you with?
I'm looking forward to your good news :).

Enjoy your day!

Jeroen