Repository navigation
fix(vertex/anthropic): passing beta header only for structured outputs - #11015
Conversation
|
all the commits till here fix the original issue. commits after this point are to account for the change in example that example will fail and give an error. but to enable it for users who want to use the latest anthropic models with vertex, we force the use of JSON tool |
lgrammel
left a comment
There was a problem hiding this comment.
code looks good but test changes indicate a regression might have been introduced
expected behaviour, see comment above! |
gr2m
left a comment
There was a problem hiding this comment.
code changes look good but haven't tested end-to-end, I assume you did
not with UI example (there isn't one for vertex or structured outputs) but yes, tested with different variations of examples (passing tools, tools + object output) |
| responseFormat?.type === 'json' && | ||
| responseFormat.schema != null; |
There was a problem hiding this comment.
why is this additional condition needed?
There was a problem hiding this comment.
just as an extra check - to completely rule out header being sent for functions that don't need it. otherwise it could send it for simple text gen too. i verified with expected behaviour for approx 6 different cases (combinations essentially)
|
This also seems to fix a bug around strict mode and setting the header. Can this either be separated out into a different PR or added to the description? |
EDIT: added in the description |
|
@aayush-kapoor why it is required to send the structured output header when strict mode is off ? |
Background
Currently, the vertex anthropic provider only checked the model ID and would pass the beta header for all models that allow structured outputs.
But vertex doesn't yet support passing these beta headers. So we would get an error like this even for a simple generateText function
This PR also addresses a bug when strict mode is enabled and the header is passed (shouldn't happen)
Summary
Manual Verification
the example
google-vertex-anthropic-sonnet-4-5.tswas used to verify the fix (was failing before)Checklist
pnpm changesetin the project root)Future Work
ideally, we handle this shortcoming in our implementation of vertex provider
structuredOutputMode: 'jsonTool'(continued in this PR after the comment fix(vertex/anthropic): passing beta header only for structured outputs #11015 (comment))Related Issues
Fixes #10981