feature: support CTranslate2

[mudler]

Among the backends that we already have, we should consider also ctransformers2 which is high performance: https://opennmt.net/CTranslate2/generation.html besides #796

[localai-bot]

⚠️⚠️⚠️⚠️⚠️

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful 😸 and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me,
I don't support (yet) replying!

⚠️⚠️⚠️⚠️⚠️

Great suggestion! We can definitely consider adding support for ctransformers2 as it seems to be a powerful backend. I will add this to our short-term roadmap and we will investigate the feasibility of implementing it. Thank you for bringing this up!

Sources:

• https://localai.io/index.html
• https://localai.io/model-compatibility/index.html
• https://localai.io/basics/news/index.html

[Aisuko]

@mudler Show more detail please. Do you mean we use it on Text generation or encode?

[bgeneto]

@mudler Show more detail please. Do you mean we use it on Text generation or encode?

I think for Speech-to-text usage with e.g. Systran/faster-whisper-large-v3 model. It could be another backend for Audio to text besides whisper.cpp