fix(core): warn when a preview model is silently substituted - #28828
fix(core): warn when a preview model is silently substituted#28828chelsealong wants to merge 2 commits into
Conversation
When a saved/requested model is a preview model (e.g. gemini-3.1-pro-preview) but the authenticated account has no preview access, Config silently rewrites the active model to the auto alias with no indication to the caller. Log a warning naming the requested model so the substitution is visible instead of silent. Fixes google-gemini#28825
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses an issue where the system would silently fall back to a default model when a user requested a preview model they were not entitled to use. By introducing a warning message, the system now provides necessary transparency to the user when their requested model is automatically substituted, improving debuggability and user awareness. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
📊 PR Size: size/M
|
There was a problem hiding this comment.
Code Review
This pull request introduces a warning log when a requested preview model is substituted with a default model due to a lack of account entitlement, and adds a corresponding unit test to verify this behavior. The feedback suggests restoring the debugLogger.warn spy at the end of the new test to prevent potential test leakage, as debugLogger is a shared module-level singleton.
| expect(warnSpy).toHaveBeenCalledWith( | ||
| expect.stringContaining(PREVIEW_GEMINI_3_1_MODEL), | ||
| ); | ||
| }); |
There was a problem hiding this comment.
The spy on debugLogger.warn is not restored at the end of the test. Since debugLogger is a shared module-level singleton, not restoring the spy can leak into other tests in the suite, potentially swallowing warnings or causing unexpected side effects. Please call warnSpy.mockRestore() at the end of the test to clean up the spy, matching the pattern used elsewhere in this file.
| expect(warnSpy).toHaveBeenCalledWith( | |
| expect.stringContaining(PREVIEW_GEMINI_3_1_MODEL), | |
| ); | |
| }); | |
| expect(warnSpy).toHaveBeenCalledWith( | |
| expect.stringContaining(PREVIEW_GEMINI_3_1_MODEL), | |
| ); | |
| warnSpy.mockRestore(); | |
| }); |
…n test Prevents the spy from leaking into other tests in the shared config.test.ts suite, per review feedback.
|
Addressed: restored the |
Summary
Fixes #28825.
When a user requests a preview model (e.g.
gemini-3.1-pro-preview) but theirauthenticated account has no preview-model entitlement,
Configsilentlyrewrites the active model to the
auto-gemini-2.5alias with zeroindication to the caller — no error, no warning, nothing in the CLI output.
The substitute model is then chosen by the auto-routing/classifier logic,
which is why different callers can observe different substitutes
(
gemini-2.5-provsgemini-2.5-flash) for the same requested model,as described in the issue.
This is a minimal fix for the smallest actionable ask in the issue ("Failing
that, warn on stderr whenever the served model differs from the requested
one."). The larger asks (hard error,
--strict-modelflag) require across-component refactor across
packages/cliargument parsing and multiplepackages/corefallback/routing components — the same scope called out aseffort/mediumand requiring further design in the related, closed issue#26938. This PR keeps things scoped to the specific code path this report
demonstrates:
Config's post-auth preview-access check(
packages/core/src/config/config.ts), which silently resetthis.modelviasetModel(DEFAULT_GEMINI_MODEL_AUTO)with no log of any kind.Change
In
packages/core/src/config/config.ts, when this reset fires, log awarning (via the existing
debugLogger.warn, which surfaces to stderr innon-interactive mode and to the console message log in interactive mode)
naming the originally-requested model:
Test plan
Added a regression test in
packages/core/src/config/config.test.ts(
Model Persistence Bug Fix (#19864)suite) that:Configwith the initial model set to the Gemini 3.1preview model.
getCodeAssistServer/retrieveUserQuotaso the account's quotabuckets do not include a preview model (i.e. no entitlement).
refreshAuth(AuthType.LOGIN_WITH_GOOGLE)(the personal OAuth flowfrom the issue's repro).
DEFAULT_GEMINI_MODEL_AUTO(existingbehavior, unchanged) and that
debugLogger.warnwas called with amessage naming the originally-requested model (the new behavior).
Verified the test fails without the fix:
Full suite run after the fix:
Also ran, with no new errors:
AI assistance disclosure
This change was implemented with the assistance of an AI coding agent
(Claude, Anthropic), which located the root cause, wrote the fix, and wrote
and verified the regression test described above.