More and better options for text-to-speech voices

[klues]

Recently our ResponsiveVoice integration stopped working because we're above their "fair use" limit. It never was very reliable, so no problem to get rid of it.

However, for some cooperations we need more reliable options for text-to-speech (TTS), especially for Linux clients (e.g. hospital terminals), where there are no offline system voices available.

Since there are many possible providers (online services like MS Azure, Google, Polly etc.) and maybe also new forms of in-browser offline voices (e.g. test piper.ttstool.com - I think we need some adaptions to the UI for selecting voices. Probably it shouldn't be a single dropdown (with potentially hundreds of voices) anymore, but some 2-step selection.

UI proposal:

These would be possibilities for voice providers:

• System voices: default - like now - voices coming from WebSpeech API (voices installed on the system)
• online services like MS Azure, Amazon Polly etc.
• voices coming from the external speech bridge I've once implemented (which will somehow be replaced by in-Asterics implementations then, but for special cases maybe still valuable)

For services which need configuraiton (like API keys), there will be the Configure button next to the select, which opens a modal.

@arasaac-dga @ms-mialingvo @willwade @ChrisVeigl - you're welcome to give your feedback on this proposal.
@willwade I'm wondering if I should use your js-tts-wrapper for implementation in the background - probably yes - but what do you think about production-readiness of it?

related: #181

[willwade]

Yeah - I think this would be good way of testing out how robust the js-tts-wrapper is. I havent used it much in the browser but we know it should work. That piper web is interesting. I know for a fact I was struggling to get the sherpaonnx wasm working so that may be a better solution.

[ms-mialingvo]

Yeah, I don't understand the technical part of what you're talking about 😅 So just commenting on the visual part and the usability.
What would be helpful for users is a clear way to set up

1. an offline voice if available (with the info-button/link that leads to the manual and tutorials on how to set additional offline voices up)
2. an online voice as backup when the offline voice fails or is not available
3. ...and that for all the languages in the grid set, if the grid set is multilingual. And including the option to nevertheless have all voices visible, when one needs an alternative voice for languages for which a TTS doesn't yet exist (e.g. French for Kreole, German for Swiss Dialect)

Other terminology like 'external speech bridge' or even 'system voice' gets confusing.
Reminder, that overall it's speech language pathologists or parents who will introduce and set up an AAC app, so people with zero knowledge in software development and any related terminology and often even only mediocre skills in handling an iPad/tablet ;)

About (1): For Android, voices bought through apps like acapela or Vocalizer show up labelled wrongly as 'online', despite being available offline. I assume that's an issue of the respective apps, not of Asterics. I don't know if that influences the possibility to filter offline voices in the dropdown.

[arasaac-dga]

Since there are many possible providers (online services like MS Azure, Google, Polly etc.) and maybe also new forms of in-browser offline voices (e.g. test piper.ttstool.com - I think we need some adaptions to the UI for selecting voices. Probably it shouldn't be a single dropdown (with potentially hundreds of voices) anymore, but some 2-step selection.

We agree. It would be neccesary a new options and dropdown to better select and configure the new providers. So, we don't see problems in the new organization you propose.

• voices coming from the external speech bridge I've once implemented (which will somehow be replaced by in-Asterics implementations then, but for special cases maybe still valuable)

Perhaps, to avoid confiussion to users and improve usability this option only must be showed when this "bridge" is configurated or activated (for example in the actual integration area where is now).

Other terminology like 'external speech bridge' or even 'system voice' gets confusing.

We agree. Offline and Online voices is the best denomination

Reminder, that overall it's speech language pathologists or parents who will introduce and set up an AAC app, so people with zero knowledge in software development and any related terminology and often even only mediocre skills in handling an iPad/tablet ;)

We totally agree with @ms-mialingvo Day by day we found it with people that give support. So, we have to try to make everything as easy as possible por managers.

About (1): For Android, voices bought through apps like acapela or Vocalizer show up labelled wrongly as 'online', despite being available offline. I assume that's an issue of the respective apps, not of Asterics. I don't know if that influences the possibility to filter offline voices in the dropdown.

Thanks for the information @ms-mialingvo We didn't know it. It could be a problem because for some languages it's the only option to have voices in their languages. Investigate if it is a problem of the apps and if it's posible to make anything by Asterics part to better indentify them by users in the list of voices.

[ms-mialingvo]

Perhaps, to avoid confiussion to users and improve usability this option only must be showed when this "bridge" is configurated or activated (for example in the actual integration area where is now).

Makes sense to me too

[klues]

Just for more clarification, it will be like this:

• for speech provider the default setting (whatever you call it System voices or Default voices or give me any better suggestion) will be just the same as now - the same voices as now will be selectable in "Preferred voice" - it will show all voices which are directly available to the browser. These can be:
  • voices installed on the system (e.g. Microsoft voices on Windows, Google voices on Android, iOS voices, voices installed by additional apps like Acapela, ...) - these should be offline
  • voices which are provided by the browser (e.g. Chrome offers additional Google voices on Desktop, Edge browser offers online voices from MS, ...) - these are often online
  • Default system voice: which is just a hack to get the default system voice on iOS, essentially by passing nothing (null) as voice-ID to the API of the browser
• ResponsiveVoice voices will be removed for the reason already explained.

So in the best case, users will just find a suitable voice with the standard setting Default voices as voice provider. But, if there are no good voices available, there will be the possibility to select another provider (MS Azure, Polly, Piper, ...) - but for all these additional providers there will be some additional steps to configure (e.g. insert API keys, download voices for Piper, ...). This then can be done in the "configure" modal.
Therefore I think it's necessary to have this complexity of "voice provider" visible to the user, because every other provider than "Default voices" simply needs some extra configuration.

an online voice as backup when the offline voice fails or is not available

So you think of being able to select 2 preferred voices, one as default, one as backup?! I think this wouldn't make much sense, because if there is an offline voice available there is no need for a backup (offline voice will always work). Maybe it would make sense to have a backup for an online voice - but probably will confuse users even more if they have 2 voices to select on this settings page...

and that for all the languages in the grid set, if the grid set is multilingual

in the "change language" action there will still be the possibility to change to voice while changing the language, so this will stay as it is.

For Android, voices bought through apps like acapela or Vocalizer show up labelled wrongly as 'online',

Yeah, a bug of Android or Chrome where we cannot do anything about it (but maybe could file an issue there). Any difference in Firefox?

(with the info-button/link that leads to the manual and tutorials on how to set additional offline voices up)

yes there should be a link to a tutorial how to set up things for every speech provider and for the Default voices a link to a tutorial how to install additional offline voices on the system.

Offline and Online voices is the best denomination

The voices from the browser API (Default voices) can be both, offline or online. Of course we could add two categories in "Specch provider": Offline default voices, Online default voices, but I think it wouldn't make too much sense, at least as long there are still some voices which are labelled wrongly - see above. I think I would keep it as it is - order offline voices first and online voices last in the dropdown for Preferred voice.

[ms-mialingvo]

But, if there are no good voices available, there will be the possibility to select another provider (MS Azure, Polly, Piper, ...) - but for all these additional providers there will be some additional steps to configure (e.g. insert API keys, download voices for Piper, ...). This then can be done in the "configure" modal.
Therefore I think it's necessary to have this complexity of "voice provider" visible to the user, because every other provider than "Default voices" simply needs some extra configuration.

What would the other options in the dropdown show? Names of other providers will mean nothing to pretty much all users and they'd have to know in advance if that provider has the language that they need or how would it work?

Dropdown shows the already installed/available voices. The button is labelled 'configure additional voices' which is a clear label. And then, I don't know what exactly the additional steps are for each provider, but I would have to super quickly see which providers have the language I'm looking for.

So you think of being able to select 2 preferred voices, one as default, one as backup?! I think this wouldn't make much sense, because if there is an offline voice available there is no need for a backup (offline voice will always work). Maybe it would make sense to have a backup for an online voice - but probably will confuse users even more if they have 2 voices to select on this settings page...

I guess I was thinking about windows (and I don't quite remember if iOS too) where some online voices are better than some offline voices so when there's internet it would be better to use those but there needs to be a backup when there's no internet. But yeah, forget that suggestion for the moment as for the moment it would just add more complexity I think.

in the "change language" action there will still be the possibility to change to voice while changing the language, so this will stay as it is.

Yes, and the feedback from users is that it's confusing that the same option ( = selecting a voice) is on such different places. (It also means that you can't apply the advanced voice settings pitch/speed to other voices) Not saying that this actually has to change, just talking about ideals.

Yeah, a bug of Android or Chrome where we cannot do anything about it (but maybe could file an issue there). Any difference in Firefox?

As far as I remember it was the same in Firefox.

[klues]

What would the other options in the dropdown show? Names of other providers will mean nothing to pretty much all users

Yes, names of other providers. Yes, I know, most users will not directly understand the meaning (and will need some tutorial or similar), but what is the alternative? If the default voices aren't enough we simply need another provider, which needs some extra setup. And we cannot do the setup for the users, because most providers are paid (some with free contingents) and we don't sell all-in-one packages. For free alternatives like Piper you have to try and download voices - see https://piper.ttstool.com/ - so at least the user has to decide which voice should be downloaded. So although it's somehow complex - we cannot take away these steps from the user. At least I think so, but I'm open for better ideas.

I would have to super quickly see which providers have the language I'm looking for

We can provide a link to a list provided by the provider - but I don't think we can maintain a list on our own (and keep it updated).

[willwade]

I should have something for you all to see on this front.. watch this space (@klues - the answer was a flat NO js-tts-wrapper was in a awful state. It lied about it being good in browser js envs but I think ive fixed this..)..

[willwade]

@ms-mialingvo see https://ttsvoicesavailable.streamlit.app or https://willwade.github.io/WorldAlphabets/ to see what languages supported by what TTS. The latter doesnt have any SAPI/System voices but the former does

[klues]

Just did some quick test for Piper ( https://piper.ttstool.com/ ) - voice generation is too slow on standard Samsung smartphone and on iPad it doesn't work at all. So I think we can drop it for now (unfortunately).

[ms-mialingvo]

but what is the alternative?

I already wrote a suggestion.

And since I don't know how the setup for the providers is so have no clue how complex the setup is I can't give a proper idea for how the modal upon clicking 'configure additional voices' should look like but something along this I guess

With the 'system' tab only having some basic written info and links to the manual/videos and the 'bridge' tab having the setup for External speech service.

And once configured the voices would show up in the dropdown.

Do voices get configured or downloaded individually or gives a simple configuration access to all voices from that provider at once?
Also, do these providers work accross systems? (iOS, Windows, Android, Linux) or are they system-specific?
Android is quite well covered with voices that can be downloaded from apps like acapella. iOS and Windows (and I guess Linux) are the main concerns.

[willwade]

And once configured the voices would show up in the dropdown.

Do voices get configured or downloaded individually or gives a simple configuration access to all voices from that provider at once?

A LOT of voices are now online only. We have a couple of offline providers but these are tricky to get working. Lets see.. There are many ways we can filter them (Online/Offline, Scripts (NB: This isnt done much but for many of the newer tts systems you need to be careful at the script you send the tts- Some just wont work if you send latin/devangari etc. And MANY people use transliteration and arent literate in their native script. So this becomes a challenge.. But I digress), Provider, Region, Language. You could do per Voice Engine or as one big list but filtering close to 3000+ voices is difficult without some filtering

Also, do these providers work accross systems? (iOS, Windows, Android, Linux) or are they system-specific? Android is quite well covered with voices that can be downloaded from apps like acapella. iOS and Windows (and I guess Linux) are the main concerns.

Yes. In short - Yes. All of this works in the browser. Now there are many tts engines only supporting native (SAPI, AVSynth on iOS and others) BUT as we have noted elsewhere (and the point you are making) is iOS has a problem accessing those from the browser env.. But yes - many we can do in the browser.

[ms-mialingvo]

You could do per Voice Engine or as one big list but filtering close to 3000+ voices is difficult without some filtering

Filtering would happen as is already the case now by filtering language first which limits the options shown in preferred voice

Or am I understanding something wrong?

[willwade]

No you arent missing anything. English is a good one though. You end up with like a 1000 voices for English.. that might be not enough to filter on just alone with English.. Thats my point (its unlikely to be 1000 - thats the case if you add several engines..) anyway - wait a moment- it will be easier If I can demo this..

[ms-mialingvo]

Why would you add any extra voices when you already have enough options to choose from without any additional installing as is the case for English?

(And my point still would stand, using voice provider as an additional filter when the names of the voice provider literally doesn't tell anything to 99.99% of the users doesn't make a difference vs the 1000 voices, it just becomes a random filter).