feat(gemini): add opt-in support for responseJsonSchema #18147
+297
−20
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add support for Gemini's native responseJsonSchema parameter which uses
standard JSON Schema format instead of OpenAPI-style responseSchema.
Benefits of responseJsonSchema (Gemini 2.0+ only):
- Standard JSON Schema format (lowercase types)
- Supports additionalProperties for stricter validation
- Better compatibility with Pydantic's model_json_schema()
- No propertyOrdering required
Usage:
```python
response_format={
"type": "json_schema",
"json_schema": {"schema": {...}},
"use_json_schema": True # opt-in
}
```
This is backwards compatible - existing code continues to use
responseSchema by default.
Closes BerriAI#16340
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Document the new use_json_schema option for Gemini 2.0+ models in the JSON Mode documentation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relevant issues
Fixes #16340
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
Changes
Add opt-in support for Gemini's native
responseJsonSchemaparameter (Gemini 2.0+ models).What is
responseJsonSchema?Gemini API supports two schema formats for structured outputs:
responseSchema(OpenAPI format) - uppercase types, no additionalPropertiesresponseJsonSchema(JSON Schema format) - standard format, supports additionalPropertiesBenefits of
responseJsonSchema:string,object)additionalProperties: falsefor stricter validationmodel_json_schema()propertyOrderingrequiredUsage:
Backwards Compatible:
use_json_schema: True, behavior is unchanged (usesresponseSchema)responseSchemawith a warningFiles Changed:
litellm/types/llms/vertex_ai.py- Addedresponse_json_schemato GenerationConfiglitellm/llms/vertex_ai/common_utils.py- Addedsupports_response_json_schema()and_build_json_schema()litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py- Modified transformation to support opt-intests/.../test_vertex_and_google_ai_studio_gemini.py- Added 2 testsTest
test_vertex_ai_response_json_schema_opt_in- Verifies opt-in works for Gemini 2.0+test_vertex_ai_response_json_schema_fallback_for_old_models- Verifies fallback for older models