Metas MMS speech recognition

[flozi00]

Model description

In their Blogpost they are writing it's 1B sized wav2vec 2 models, so only a new converter script should be needed?

A nice alternative to compare to whisper

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

https://github.com/facebookresearch/fairseq/tree/main/examples/mms

@sanchit-gandhi
@patrickvonplaten FYI

[JoesSattes]

Interested Model!! Thank @flozi00 to open request.

[NielsRogge]

👀 https://huggingface.co/models?other=mms

[sorenmulli]

👀 https://huggingface.co/models?other=mms

Nice! This is the pretrained model.
I am looking to also convert the ASR models; any pointers?

The scripts I am looking at (such as this very useful boi) require dict and HF config but maybe it is the same configuration as facebook/wav2vec2-large 🤔

[patrickvonplaten]

Leeez go - working on the conversion as we speak :-)

[LYPinASR]

I also made a request in #23811. Looking forward to it!

[patrickvonplaten]

PR merged.

Also see:

• https://huggingface.co/docs/transformers/main/en/model_doc/mms
• [MMS] Scaling Speech Technology to 1,000+ Languages | Add attention adapter to Wav2Vec2 #23813
• https://huggingface.co/facebook/mms-1b-all