Skip to content

fix(vertex/anthropic): passing beta header only for structured outputs - #11015

Merged
lgrammel merged 16 commits into
mainfrom
aayush/10981-vertex-structured-outputs
Dec 16, 2025
Merged

lgrammel merged 16 commits into
mainfrom
aayush/10981-vertex-structured-outputs

Conversation

@aayush-kapoor

@aayush-kapoor aayush-kapoor commented Dec 9, 2025 •

Copy link
Copy Markdown
Collaborator

Background

Currently, the vertex anthropic provider only checked the model ID and would pass the beta header for all models that allow structured outputs.

But vertex doesn't yet support passing these beta headers. So we would get an error like this even for a simple generateText function

This PR also addresses a bug when strict mode is enabled and the header is passed (shouldn't happen)

Summary

  • only pass the beta header when actually using native output format and when using strict tools

Manual Verification

the example google-vertex-anthropic-sonnet-4-5.ts was used to verify the fix (was failing before)

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)

Future Work

ideally, we handle this shortcoming in our implementation of vertex provider

Related Issues

Fixes #10981

@ghost ghost added ai/provider related to a provider package. Must be assigned together with at least one `provider/*` label provider/anthropic Issues related to the @ai-sdk/anthropic provider labels Dec 9, 2025
@aayush-kapoor

Copy link
Copy Markdown
Collaborator Author

all the commits till here fix the original issue.

commits after this point are to account for the change in example /examples/ai-core/src/generate-object/google-vertex-anthropic.ts which fails in the current flow

that example will fail and give an error. but to enable it for users who want to use the latest anthropic models with vertex, we force the use of JSON tool

Comment thread examples/ai-core/src/generate-object/google-vertex-anthropic-sonnet-4-5.ts Outdated
Comment thread examples/ai-core/src/generate-object/google-vertex-anthropic.ts
Comment thread packages/anthropic/src/anthropic-messages-language-model.test.ts Outdated
Comment thread packages/anthropic/src/anthropic-messages-language-model.test.ts Outdated
Comment thread examples/ai-core/src/generate-object/google-vertex-anthropic-sonnet-4-5.ts Outdated

@lgrammel lgrammel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks good but test changes indicate a regression might have been introduced

@aayush-kapoor

Copy link
Copy Markdown
Collaborator Author

code looks good but test changes indicate a regression might have been introduced

expected behaviour, see comment above!

@gr2m gr2m left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code changes look good but haven't tested end-to-end, I assume you did

Comment thread .changeset/lemon-sheep-behave.md Outdated
Ensure beta header is passed only for structured outputs in vertex and anthropic.
@aayush-kapoor

aayush-kapoor commented Dec 12, 2025 •

Copy link
Copy Markdown
Collaborator Author

code changes look good but haven't tested end-to-end, I assume you did

not with UI example (there isn't one for vertex or structured outputs) but yes, tested with different variations of examples (passing tools, tools + object output)

Comment on lines +492 to +493
responseFormat?.type === 'json' &&
responseFormat.schema != null;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this additional condition needed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just as an extra check - to completely rule out header being sent for functions that don't need it. otherwise it could send it for simple text gen too. i verified with expected behaviour for approx 6 different cases (combinations essentially)

@lgrammel

Copy link
Copy Markdown
Collaborator

This also seems to fix a bug around strict mode and setting the header. Can this either be separated out into a different PR or added to the description?

@aayush-kapoor

aayush-kapoor commented Dec 15, 2025 •

Copy link
Copy Markdown
Collaborator Author

This also seems to fix a bug around strict mode and setting the header. Can this either be separated out into a different PR or added to the description?

EDIT: added in the description

@lgrammel
lgrammel merged commit 87db851 into main Dec 16, 2025
18 checks passed
@lgrammel
lgrammel deleted the aayush/10981-vertex-structured-outputs branch December 16, 2025 10:57
@AVtheking

Copy link
Copy Markdown
Contributor

@aayush-kapoor why it is required to send the structured output header when strict mode is off ?

This branch was successfully deployed

1 active deployment
Preview — ac2a10b1 Deployed Dec 15, 2025 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai/provider related to a provider package. Must be assigned together with at least one `provider/*` label provider/anthropic Issues related to the @ai-sdk/anthropic provider task-merge-tracking-done

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vertex: Unexpected value(s) structured-outputs-2025-11-13 for the anthropic-beta header

4 participants