feat: wire Vercel AI Gateway reasoning via AI SDK v6 native path#11449
feat: wire Vercel AI Gateway reasoning via AI SDK v6 native path#11449hannesrudolph wants to merge 1 commit intomainfrom
Conversation
- Parse model tags from /v1/models for reasoning capability detection - Use top-level 'reasoning' param in streamText/generateText (AI SDK v6) - Add providerOptions.anthropic.thinking for Anthropic models through Gateway - Use computed reasoningBudget (80% cap) for thinking budget tokens - Add test coverage for reasoning + anthropic thinking payload shape
Reviewed all 4 changed files. Found 2 items to address:
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
| const anthropicProviderOptions = | ||
| modelId.startsWith("anthropic/") && reasoningConfig | ||
| ? { | ||
| anthropic: { | ||
| thinking: { | ||
| type: "enabled" as const, | ||
| budgetTokens: reasoningBudget ?? Math.floor((info.maxTokens ?? 0) * 0.8), | ||
| }, | ||
| }, | ||
| } | ||
| : undefined |
There was a problem hiding this comment.
Anthropic requires temperature = 1.0 when extended thinking is enabled. The existing getModelParams enforces this for budget-based models (via shouldUseReasoningBudget), but Gateway Anthropic models only have supportsReasoningEffort: true -- they lack supportsReasoningBudget -- so the budget path is never entered and the temperature stays at 0.7 (VERCEL_AI_GATEWAY_DEFAULT_TEMPERATURE). If the Vercel AI Gateway does not transparently override the temperature for Anthropic thinking requests, this will cause the API to reject the call. The same applies to the identical block in completePrompt. Consider forcing resolvedTemperature = 1.0 when anthropicProviderOptions is defined (and no explicit user temperature is set).
Fix it with Roo Code or mention @roomote and request a fix.
| const reasoningConfig = reasoning ? { enabled: true, effort: reasoning.reasoning_effort } : undefined | ||
| const anthropicProviderOptions = | ||
| modelId.startsWith("anthropic/") && reasoningConfig | ||
| ? { | ||
| anthropic: { | ||
| thinking: { | ||
| type: "enabled" as const, | ||
| budgetTokens: reasoningBudget ?? Math.floor((info.maxTokens ?? 0) * 0.8), | ||
| }, | ||
| }, | ||
| } | ||
| : undefined |
There was a problem hiding this comment.
This reasoningConfig + anthropicProviderOptions block is identical to the one in createMessage (lines 163-174). Extracting both into a small private helper (e.g. buildReasoningOptions(modelId, info, reasoning, reasoningBudget)) would eliminate the duplication and ensure future fixes (like the temperature override) only need to be applied once.
Fix it with Roo Code or mention @roomote and request a fix.
Summary
Properly configures reasoning support for the Vercel AI Gateway provider using AI SDK v6 semantics, replacing the previous incomplete wiring.
Changes
Model capability detection (
src/api/providers/fetchers/vercel-ai-gateway.ts)tagsarray from the/v1/modelsresponse schema"reasoning"tag (in addition to model ID prefix heuristics)supportsReasoningEffort: trueon models with reasoning tagsRequest wiring (
src/api/providers/vercel-ai-gateway.ts)reasoningparameter instreamText()andgenerateText()anthropic/\*), additionally passproviderOptions.anthropic.thinkingwith{ type: "enabled", budgetTokens }— the documented way to enable Claude extended thinking through GatewayreasoningBudget(80% of maxTokens cap) with fallbackreasoningBudgettoModelSelectiontypeTests (
src/api/providers/__tests__/vercel-ai-gateway.spec.ts)reasoningandproviderOptions.anthropic.thinkingare sent when reasoning is enabled on an Anthropic modelFetcher tests (
src/api/providers/fetchers/__tests__/vercel-ai-gateway.spec.ts)parseVercelAiGatewayModel()Why
The Vercel AI Gateway uses the OpenAI-compatible API format, but Anthropic reasoning through Gateway requires explicit
providerOptions.anthropic.thinkingconfiguration per AI SDK v6 docs. Without this, the model computes reasoning internally but does not return thinking blocks in the stream.Validation
cd src && npx vitest run api/providers/__tests__/vercel-ai-gateway.spec.ts api/providers/fetchers/__tests__/vercel-ai-gateway.spec.ts— 32/32 passing