Support for NVIDIA Parakeet models

[serovar]

Adding NVIDIA’s Parakeet (v3) as an optional backend would be a welcomed feature, both for the better performance, both to keep Buzz model-agnostic and future-proof by giving users a choice of modern ASR families beyond Whisper.

[raivisdejus]

Happy to implement this as future feature if there is any meaningful benefit. In my tests for Latvian the Parakeet model is much worse than available Whisper models. Benchmark tables in model cards say it should be better, but in reality it is worse.

If community has examples where Parakeet is better, I will definitely look into implementing support for it

[serovar]

I agree Parakeet isn’t always better (Latvian is a known “weak spot”, and the v3 model card itself shows high WER for it), but for European languages it has a strong accuracy (e.g., Italian WER 3.0% on FLEURS, 10.1% on MLS and 3.7% on CoVoST), and in my own tests (in Italian), it’s a bit better the Whisper Large v2 and v3.

Moreover, and maybe more importantly, it’s less resource intensive, since the multilingual v3 is 600M params vs Whisper large-v3 at 1.55B.

In practice I think it would be a good alternative for the languages where it’s stronger or for the users with less performant hardware, while keeping Whisper where it wins (like Latvian).