catalan language image

[JarbasAl]

Below are the current steps needed to make an image tuned for catalan language

Bundled Plugins

• https://github.com/JarbasLingua/jarbas-stt-plugin-chromium to ensure STT works since selene supports non-english langs "by accident" and it might be deprecated
• https://github.com/JarbasLingua/jarbas-wake-word-plugin-snowboy for "wake up" hotword
• https://github.com/JarbasLingua/jarbas-tts-plugin-voicerss
• https://github.com/JarbasLingua/jarbas-tts-plugin-catotron
• https://github.com/JarbasLingua/jarbas-wake-word-plugin-vosk + catalan model from https://github.com/JarbasLingua/iberian-vosk
• https://github.com/HelloChatterbox/dummy_wakeword_plugin to allow disabling wakewords (should be a default in every image imho )

Additional dependencies

• festival apt-get -y install festival festvox-ca-ona-hts lame
• maybe snowboy, plugin includes pre-compiled binaries this will need testing, https://github.com/JarbasLingua/jarbas-wake-word-plugin-snowboy/blob/master/compiling.md
• vlc for proper https streams handling

Skill alternatives

For official skills not supporting catalan

• https://github.com/JarbasSkills/skill-icanhazdadjoke
• https://github.com/JarbasSkills/skill-duck
• https://github.com/JarbasSkills/skill-wolfie
• https://github.com/JarbasLingua/skill-news

Default config

under /etc/mycroft/mycroft.conf

{
  "lang": "ca-es",
  "stt": {
      "module": "chromium_stt_plug",
      "chromium_stt_plug": {
          "lang": "ca-ES"
      },
      "vosk_streaming_stt_plug": {
          "model_folder": "path/to/bundled/catalan/model/folder"
      },
      "vosk_stt_plug": {
          "model_folder": "path/to/bundled/catalan/model/folder"
      }
   },
  "tts": {
      "module": "festival",
      "festival": {
          "lang": "catalan",
          "encoding": "ISO-8859-15//TRANSLIT"
      },
      "voicerss_tts_plug": {
          "voice": "Rut",
          "lang": "ca-es"
      }
   },
  "skills": {
    "blacklisted_skills": [
          "mycroft-npr-news.mycroftai", 
          "mycroft-fallback-duck-duck-go.mycroftai", 
          "mycroft-joke.mycroftai",
          "fallback-wolfram-alpha.mycroftai"
    ]
  },
  "listener": {
      "stand_up_word": "desperta"
  },
  "hotwords": {
     "desperta": {
        "module": "snowboy_ww_plug",
        "models": [
            {"sensitivity": 0.5, "model_path": "desperta_jm.pmdl"},
            {"sensitivity": 0.5, "model_path": "desperta_jm2.pmdl"}
         ]
      },
      "no_wake_word": {
        "module": "dummy_ww_plug"
      }
   },
  "Audio": {
    "backends": {
      "vlc": {
        "active": true
      }
    },
    "default-backend": "vlc"
  }

}

[JarbasAl]

we could also bundle the following, which im not sure makes sense due to low quality, but if we want to be comprehensive and bundle all catalan options

• espeak, which also supports catalan
• https://github.com/JarbasLingua/jarbas-tts-plugin-softcatala same as festival but remote
• responsive_voice plugin, supports catalan via espeak, https://github.com/JarbasLingua/jarbas-tts-plugin-responsivevoice
• https://github.com/JarbasLingua/jarbas-stt-plugin-pocketsphinx with catalan model from https://github.com/JarbasLingua/iberian-sphinx

additions to .conf in this case

"stt": {
    "pocketsphinx_stt_plug": {
        "acoustic-model": "path/to/bundled/catalan/hmm/folder",
        "language-model": "path/to/bundled/catalan/lm/file.lm",
        "pronounciation-dictionary": "path/to/bundled/catalan/phonemes/file.dict"
    }
  },
"tts": {
    "responsivevoice_tts_plug": {
        "voice": "CatalanMale"
    },
    "espeak": {
        "lang": "ca"
    }
 }

[JarbasAl]

just leaving here the link for the translation guide for catalan https://github.com/JarbasAl/mycroft-catalan.conf

[ccoreilly]

I wanted to share that at Softcatalà we have started our own OVOS image for Catalan (see the catalan branch of my fork). I am not sure what you had in mind regarding the structure for each language, for now I just created a defconfig for catalan.

We are building OVOS in a self-hosted gitlab runner and after ~12 hours it successfully builds the image. We are now debugging some issues with the image and still need to update the skills so only those with catalan support are added. Moreover we want to have STT and TTS locally so are using the vosk plugin for STT and have the major challenge of creating a festival buildroot package (it existed in buildroot but was broken since 2008 and removed in 2011) to support TTS.

Let me know if you think the language-specific file structure should be different so I can adapt it before opening a PR (this will however take a while)

[JarbasAl]

this sounds awesome!

you might want to update the vosk plugin again for https://github.com/OpenVoiceOS/ovos-stt-plugin-vosk

[ChanceNCounter]

imo, the STT and TTS options provided at first boot should be consistent between languages, to the fullest possible extent.

Also, is it worth having different images for different languages? Why not just implement proper language settings?

[ccoreilly]

I guess it depends on the use-case and target user. Moreover, there is no STT/TTS combination supporting every language.

For our current project we are aiming at a seamless plug'n'play experience. The user should only need to know how to assemble a rpi4 and a respeaker hat and how to burn the image with a simple tool like Etcher. Being able to ssh into the machine and run commands should not be necessary. Without a custom image this is not possible.

I guess generally speaking, as a goal for OVOS, a language setting + wifi configuration on first-run (via the same captive portal?) like it is done with commercial assistants might be the best option. On language selection further packages might have to be installed but I think this would be achievable with the "system" skills path you have in mind? Like if the configuration is set to using festival as TTS and it is not already installed, it should download it and install it.

[ChanceNCounter]

For our current project we are aiming at a seamless plug'n'play experience. The user should only need to know how to assemble a rpi4 and a respeaker hat and how to burn the image with a simple tool like Etcher. Being able to ssh into the machine and run commands should not be necessary. Without a custom image this is not possible.

I don't see how the one leads to the other.

I guess generally speaking, as a goal for OVOS, a language setting + wifi configuration on first-run (via the same captive portal?) like it is done with commercial assistants might be the best option

We already have wifi configuration on first run, akin to the Mk2's. Also STT and TTS selection. That's where I was imagining it, probably right in between. "English / español / francais / deutsche / ..." seems like a friendly enough touch menu to encounter during setup and onboarding. Doesn't need much context or explanation. Maybe a dynamic header that says "Select language" in various languages.

My point is, I don't think an image for a language lines up with user expectations (plus it will get to be a lot.) I'm not sure I've ever seen a French version of an operating system, or "the Cantonese" installer for a program. Rather, it's one of the first two configuration options you see in a typical setup menu, right before or after setting the time zone.

[ccoreilly]

I don't see how the one leads to the other.

Well, if you use the english OVOS image and want to use mycroft in Catalan you need to ssh into the machine in order to change the mycroft.conf and install extra plugins that don't exist currently in the image. Maybe in the future it'll be possible but currently it is not.

I'm not sure I've ever seen a French version of an operating system, or "the Cantonese" installer for a program

Not usual because English is the dominant language in technology but there are precedents.

However, it'd be great to have language selection eventually but it'll take a while so we'll try to maintain a Catalan image until then.

[JarbasAl]

i think having a catalan image gives us a good starting point when we finally get to lang selection during onboarding, i dont see it as wasted work but rather as needed research before the next steps

[JarbasAl]

maybe its a good time to start a discussion just about language selection/switching, lots of ways to go about it, the sooner we have a conversation on it the better, i will see if i can write something up tomorrow to get that ball rolling

[ccoreilly]

I agree. I am also not sure how strings in the UI can be localized or if it possible, for example.

[ChanceNCounter]

Those few strings which can't be fed dynamically to the relevant elements can probably refer to separate resource files. QML isn't heavy.

[JarbasAl]

i would love to just have a set of configs per language and change the defaults based on user input, could be added to the pairing step where we select backend and stt already

a trickier issue is prompting the user (in the voice only scenario / no gui) without already knowing the language

i think for now an image per language is good, but in the future we should try to unify this into a config/onboarding step, it goes back to the big hypothetical "hey mycroft, change language" algorithm

[JarbasAl]

@chance i think there is value in dedicated images, regional/lang specific skill are a big part of it, it also reaches a larger community

things like wolfram alpha only work in english, there is also for example a story tales skill only in catalan

having a dedicated image for that community is not a bad thing and easy to add on top of the OVOS recipe

Until we can provide a unified experience that includes this sort of thing (default wake word, skills, config) for every language i think this is the best we can do

[ChanceNCounter]

@Jarbas doesn't it strike you as the same amount of work for a less desirable goal? it does me.

[ChanceNCounter]

As for skills, they could fairly trivially be parsed for the right language codes. No dialog, no vocab, not gonna work in your language.

[JarbasAl]

if we need to bundle STT/TTS for every possible language the image will be super bloated, the same goes for skills, its not about skills not working, its about possibly being a completely different set of skills/plugins.

I dont think its duplicating work, before you can select a language you need to have it functional, i see this as research for when we actually support language selection

I do agree long term a single image that allows selecting a language is a better approach, but even after we have lang selection implemented i still don't see an issue with a dedicated image existing, especially when someone is maintaining and putting in the work for it