Skip to content

feat: wire Vercel AI Gateway reasoning via AI SDK v6 native path#11449

Open
hannesrudolph wants to merge 1 commit intomainfrom
feat/vercel-ai-gateway-reasoning-v6
Open

feat: wire Vercel AI Gateway reasoning via AI SDK v6 native path#11449
hannesrudolph wants to merge 1 commit intomainfrom
feat/vercel-ai-gateway-reasoning-v6

Conversation

@hannesrudolph
Copy link
Collaborator

Summary

Properly configures reasoning support for the Vercel AI Gateway provider using AI SDK v6 semantics, replacing the previous incomplete wiring.

Changes

Model capability detection (src/api/providers/fetchers/vercel-ai-gateway.ts)

  • Parse tags array from the /v1/models response schema
  • Detect reasoning capability from "reasoning" tag (in addition to model ID prefix heuristics)
  • Set supportsReasoningEffort: true on models with reasoning tags

Request wiring (src/api/providers/vercel-ai-gateway.ts)

  • Use AI SDK v6 top-level reasoning parameter in streamText() and generateText()
  • For Anthropic models (anthropic/\*), additionally pass providerOptions.anthropic.thinking with { type: "enabled", budgetTokens } — the documented way to enable Claude extended thinking through Gateway
  • Compute thinking budget from reasoningBudget (80% of maxTokens cap) with fallback
  • Add reasoningBudget to ModelSelection type

Tests (src/api/providers/__tests__/vercel-ai-gateway.spec.ts)

  • New test: verifies both top-level reasoning and providerOptions.anthropic.thinking are sent when reasoning is enabled on an Anthropic model
  • Model cache mock extended with a reasoning-capable Anthropic model entry

Fetcher tests (src/api/providers/fetchers/__tests__/vercel-ai-gateway.spec.ts)

  • Tests for tag-based reasoning detection in parseVercelAiGatewayModel()

Why

The Vercel AI Gateway uses the OpenAI-compatible API format, but Anthropic reasoning through Gateway requires explicit providerOptions.anthropic.thinking configuration per AI SDK v6 docs. Without this, the model computes reasoning internally but does not return thinking blocks in the stream.

Validation

  • cd src && npx vitest run api/providers/__tests__/vercel-ai-gateway.spec.ts api/providers/fetchers/__tests__/vercel-ai-gateway.spec.ts — 32/32 passing
  • Lint passes

- Parse model tags from /v1/models for reasoning capability detection
- Use top-level 'reasoning' param in streamText/generateText (AI SDK v6)
- Add providerOptions.anthropic.thinking for Anthropic models through Gateway
- Use computed reasoningBudget (80% cap) for thinking budget tokens
- Add test coverage for reasoning + anthropic thinking payload shape
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. Enhancement New feature or request labels Feb 13, 2026
@roomote
Copy link
Contributor

roomote bot commented Feb 13, 2026

Rooviewer Clock   See task

Reviewed all 4 changed files. Found 2 items to address:

  • Temperature not forced to 1.0 for Anthropic thinking -- When providerOptions.anthropic.thinking is sent with type: "enabled", Anthropic requires temperature = 1.0. Gateway models only have supportsReasoningEffort (no budget flags), so getModelParams does not override the temperature; it stays at 0.7. This may cause API rejections unless the Gateway corrects it transparently.
  • Duplicated reasoning config logic -- The reasoningConfig + anthropicProviderOptions computation (~12 lines) is copy-pasted between createMessage and completePrompt. Consider extracting into a private helper.

Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.

Comment on lines +164 to +174
const anthropicProviderOptions =
modelId.startsWith("anthropic/") && reasoningConfig
? {
anthropic: {
thinking: {
type: "enabled" as const,
budgetTokens: reasoningBudget ?? Math.floor((info.maxTokens ?? 0) * 0.8),
},
},
}
: undefined
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anthropic requires temperature = 1.0 when extended thinking is enabled. The existing getModelParams enforces this for budget-based models (via shouldUseReasoningBudget), but Gateway Anthropic models only have supportsReasoningEffort: true -- they lack supportsReasoningBudget -- so the budget path is never entered and the temperature stays at 0.7 (VERCEL_AI_GATEWAY_DEFAULT_TEMPERATURE). If the Vercel AI Gateway does not transparently override the temperature for Anthropic thinking requests, this will cause the API to reject the call. The same applies to the identical block in completePrompt. Consider forcing resolvedTemperature = 1.0 when anthropicProviderOptions is defined (and no explicit user temperature is set).

Fix it with Roo Code or mention @roomote and request a fix.

Comment on lines +227 to +238
const reasoningConfig = reasoning ? { enabled: true, effort: reasoning.reasoning_effort } : undefined
const anthropicProviderOptions =
modelId.startsWith("anthropic/") && reasoningConfig
? {
anthropic: {
thinking: {
type: "enabled" as const,
budgetTokens: reasoningBudget ?? Math.floor((info.maxTokens ?? 0) * 0.8),
},
},
}
: undefined
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reasoningConfig + anthropicProviderOptions block is identical to the one in createMessage (lines 163-174). Extracting both into a small private helper (e.g. buildReasoningOptions(modelId, info, reasoning, reasoningBudget)) would eliminate the duplication and ensure future fixes (like the temperature override) only need to be applied once.

Fix it with Roo Code or mention @roomote and request a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant