Repository navigation
Python: Honor the configured model in streamed Gemini chunks - #9139
Ayoub (Ayoubhm07) wants to merge 3 commits into
Conversation
_process_chunk passed chunk.model_version straight into ChatResponseUpdate.model, while _process_generate_response falls back to self.model for the same field 33 lines above. Both take the same types.GenerateContentResponse, and model_version is Optional[str] in the pinned google-genai. ChatResponse.model is built only from the updates, so a stream whose chunks carry no model_version ended up with model=None. The observability layer reads that field for gen_ai.response.model, which is a metric dimension of the token and duration histograms, so those lost their per-model breakdown on streams. Apply the same expression the non-streaming path already uses. A chunk that carries model_version is untouched.
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The fallback mislabels responses when request options override the client’s default model.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Adds Gemini streaming model fallback to preserve response metadata and observability dimensions.
Changes:
- Falls back to the client model when streamed chunks omit
model_version. - Adds tests for pass-through and fallback behavior.
| File | Description |
|---|---|
_chat_client.py |
Adds streamed model fallback. |
test_gemini_client.py |
Tests streamed model handling. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
_process_chunk fell back to self.model, which is the client default. A call that overrides the model through options was then labeled with the client default instead of the model that actually ran, including in the observability metrics. _prepare_request already resolves the model and it was in scope at the call site, so _process_chunk now takes it and uses it as the fallback. Adds a streaming test with an options override.
|
I added
|
…odel-fallback # Conflicts: # python/packages/gemini/agent_framework_gemini/_chat_client.py
|
Resolved the conflict with main. The only overlap was the |

Motivation & Context
_process_chunkpasseschunk.model_versionstraight intoChatResponseUpdate.model, while_process_generate_responsefalls back toself.modelfor the same field 33 lines above._types.py:2229-2230keeps the last non-Noneupdate value and the Gemini client has no catch-up afterwards, so a stream whose chunks carry nomodel_versionends up withChatResponse.model is Nonewhere the non-streaming path reports the model.observability.py:3551-3552setsgen_ai.response.modelfrom that field, andRESPONSE_MODELis inGEN_AI_METRIC_ATTRIBUTES(:3558):_filter_metric_attributesdrops absent keys before_capture_responserecords the token and duration histograms (:3604-3610). On Gemini streams those lose their per-model dimension, and_backfill_request_model(:1905) cannot recover the span name either.Description & Review Guide
:1058to:1091. Both methods take the sametypes.GenerateContentResponse(:1034-1036,:1066), andmodel_versionisOptional[str]in the pinnedgoogle-genai2.27.0 the SDK documents per-chunk absence for the very next field of that class,prompt_feedback: "Sent only in the first stream chunk."model_versionis untouched. Only the update'smodelchanges, and through aggregationChatResponse.model. Two tests added next to the existing non-streaming one; only the fallback test fails onmain, the other guards the pass-through and passes already.pytest packages/gemini/tests254 passed 10 skipped,ruff checkandruff format --checkclean,pyright0 errors on the changed source,mypy packages/gemini/testsclean all at the versions inuv.lock.self.modelis the right value here: it is the client default, not the model resolved fromoptionsat:549, which_process_chunkdoes not receive. That matches:1058exactly, so I kept parity rather than changing both happy to thread the resolved model into the two parsers instead if you prefer.MistralChatClienthas the same asymmetry at:800/:830, though its SDK typesCompletionChunk.modelas required so the gap is not reachable there; happy to follow up separately.ollamahas no fallback on either path, so it is self-consistent and out of scope.Related Issue
Fixes #9138
Contribution Checklist