Conversation
There was a problem hiding this comment.
Greptile Summary
This PR addresses compatibility issues between the OpenAI SDK and Node.js runtime for streaming API calls in the wand generation feature. The change switches from using the OpenAI SDK to native fetch for making streaming requests to Azure OpenAI and OpenAI endpoints.
The key modifications include:
-
Runtime Change: Updates the API route from
'edge'to'nodejs'runtime to support features that don't work in edge environments -
Native Fetch Implementation: Replaces the OpenAI SDK with direct HTTP calls using native
fetch, providing more control over the streaming response handling -
Manual SSE Parsing: Implements custom Server-Sent Events (SSE) parsing logic to handle the streaming response format from OpenAI/Azure endpoints
-
Enhanced Error Handling: Adds a
safeStringifyhelper function to handle circular references in error objects and improves error logging with more contextual information -
Response Headers: Updates response headers to include
'Transfer-Encoding': 'chunked'for proper Node.js streaming compatibility
This change fits into the broader codebase by maintaining the same API interface while resolving underlying streaming issues. The wand generation feature appears to be a core component that generates content using AI models, and this fix ensures it works reliably across different runtime environments. The approach aligns with the existing provider pattern seen in other parts of the codebase, though it bypasses the structured provider system for this specific use case.
Confidence score: 3/5
- This PR introduces significant complexity with manual SSE parsing that could lead to edge cases around buffer handling and connection management
- Score reflects the high-risk nature of replacing well-tested SDK functionality with custom HTTP streaming logic
- Pay close attention to
apps/sim/app/api/wand-generate/route.tsfor potential streaming and parsing issues
1 file reviewed, no comments
Summary
Direct fetch on azure/openai endpoints
Type of Change
Testing
Manual
Checklist