Groq LLM

[BenWeekes]

Please add an extension to use Groq LLM
https://groq.com/
Groq has the same interface as OAI and is faster and cheaper.

[plutoless]

Please add an extension to use Groq LLM https://groq.com/ Groq has the same interface as OAI and is faster and cheaper.

i think most of its api are openai compatible so you can really just use the openai extension, while changing the baseURL

https://console.groq.com/docs/openai

[BenWeekes]

Hi, yes that is true and we can set base_url in property.json so it would be possible to make a new graph now with groq e.g. deepgram.groq.cartesia which I understand is the fastest and cheapest cascading pipline possible today.