-
Notifications
You must be signed in to change notification settings - Fork 42
Add Groq provider support #577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@dhvll is attempting to deploy a commit to the Merit Systems Team on Vercel. A member of the Team first needs to authorize it. |
| const chunks = parseSSEGPTFormat(data); | ||
|
|
||
| for (const chunk of chunks) { | ||
| if (chunk.usage !== null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if (chunk.usage !== null) { | |
| if (chunk.usage && chunk.usage !== null) { |
The code checks if (chunk.usage !== null) but chunk.usage could be undefined, which would cause a TypeError at runtime.
View Details
Analysis
TypeError in GroqProvider.handleBody() when accessing undefined usage properties
What fails: GroqProvider.handleBody() checks if (chunk.usage !== null) but crashes when chunk.usage is undefined, attempting to access chunk.usage.prompt_tokens
How to reproduce:
// When streaming chunk has missing usage property (undefined)
const chunk = { id: "test", choices: [] }; // usage property omitted
if (chunk.usage !== null) { // undefined !== null is true
chunk.usage.prompt_tokens; // TypeError: Cannot read properties of undefined
}Result: TypeError: Cannot read properties of undefined (reading 'prompt_tokens')
Expected: Should safely skip undefined usage like OpenRouterProvider.ts does with if (chunk.usage && chunk.usage !== null)
|
Thanks for making this PR! This server side support looks good, so you are pretty much done. We also need support for this in the typescript SDK so that people can actually use it. You will just need to add the groq provider. You can look how this is done for openai here then quickly add to next sdk: echo/packages/sdk/next/src/index.ts Lines 112 to 114 in dd2e7c1
Dont worry about bumping the sdk versions, our CI will do that |
21cd217
into
Merit-Systems:rfs/groq-takeover
Resolves: #523