Skip to content

fix(models): default Claude Sonnet 5.5 to its shipped medium effort - #14193

Open
zachweyland wants to merge 6 commits into
pingdotgg:mainfrom
zachweyland:fix/sonnet-5-5-default-effort
Open

zachweyland wants to merge 6 commits into
pingdotgg:mainfrom
zachweyland:fix/sonnet-5-5-default-effort

Conversation

@zachweyland

@zachweyland zachweyland commented Sep 29, 2026 •

Copy link
Copy Markdown

Problem

#14152 pointed claude-sonnet-5-5 at the sonnet-5 profile, so the picker defaults Sonnet 5.5 to High effort. That high default matches Sonnet 5 and Sonnet 4.6, but Claude Code's bundled model table ships default_effort: "medium" for claude-sonnet-5-5, same as Opus 5.5 and Haiku 5.5:

Model default_effort in 2.1.284 in 2.1.293
claude-sonnet-4-6 high high
claude-sonnet-5 high high
claude-sonnet-5-5 medium medium
claude-opus-5-5 medium medium
claude-haiku-5-5 (not present) medium

T3 passes --effort explicitly for manifest models, so selecting Sonnet 5.5 without picking an effort runs one level higher than the model's own default. #14148 independently found the same default and added the equivalent profile, but closed as a duplicate of #14152 before that difference could land.

Change

Adds a dedicated sonnet-5-5 capability profile and points claude-sonnet-5-5 at it. The profile is a copy of the current sonnet-5 profile (including #16908's fixed 1M context window), the only difference being the default effort, medium instead of high. Bumps updatedAt. No UI code changes.

Scope and approval

No prior issue or discussion. This is submitted as a focused fix: the composer's default effort for Sonnet 5.5 doesn't match the default Claude Code ships for the model. The change is limited to that one default, in one manifest profile. No other model, option, or behavior changes.

#16903 made the same correction for Haiku 5.5, which got its own profile defaulting to Medium to match Claude Code, as Opus 5.5 already had. Sonnet 5.5 is the remaining 5.5 model still sharing an older model's profile with a different default.

Evidence that Medium is the shipped default (catalog excerpt and the resulting T3 selection) is in this comment, in reply to @juliusmarminge's request.

Verification

After merging main (through #16908):

  • ModelManifest.test.ts, ClaudeModelCatalog.test.ts, and claudeModelOptions.test.ts: 3 files, 25 tests pass.
  • Every server test file matching Claude (including ClaudeAdapterV2.test.ts): 13 files, 250 tests pass.
  • vp fmt --check on the manifest is clean.
  • The 2.1.293 column above was read from the installed claude binary's bundled model table, the same way as in the linked comment.

Not checked: the composer in a running build with this manifest. The change is JSON-only, and the catalog tests cover how the profile resolves.

Original change made by Qwen3.8 Flash Next in OpenCode, running in T3 Code. Merged onto current main and description updated by Claude Opus 5.5 in Claude Code, running in T3 Code.

🤖 Generated with Claude Code

pingdotgg#14152 pointed claude-sonnet-5-5 at the sonnet-5 profile, which defaults
effort to high. Claude Code 2.1.284 ships default_effort "medium" for
claude-sonnet-5-5 (it stays "high" for claude-sonnet-5 and
claude-sonnet-4-6), so selecting Sonnet 5.5 runs hotter than the model's
own default. Add a dedicated sonnet-5-5 profile, identical to sonnet-5
except for the medium effort default, and bump updatedAt.

Changes made by Qwen3.8 Flash Next in OpenCode, running in T3 Code.
@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Sep 29, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This manifest-only change alters the default effort sent for Claude Sonnet 5.5 from High to Medium on the existing production path. Although the scope is narrow and explicit user selections remain supported, product-default changes require human review.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: eb8af6a7-1088-4e13-a661-2c3bc0984b51
📥 Commits

Reviewing files that changed from the base of the PR and between ff396f1 and f9c3757.

📒 Files selected for processing (1)
  • apps/server/src/provider/model-manifest.json

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The provider model manifest adds a Sonnet 5.5 profile with reasoning options and a fixed context window. The Claude Sonnet 5.5 model entry now uses this profile.

Changes

Sonnet 5.5 profile

Layer / File(s) Summary
Define and assign the Sonnet 5.5 profile
apps/server/src/provider/model-manifest.json
The profile adds a Reasoning selector with low, medium, high, extra-high, max, and ultrathink options. It sets a fixed 1,000,000-token context window. The model entry uses the new profile, and the manifest timestamp is updated.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: arturict, juliusmarminge

Merge Risk: ⚪ Minimal · up to f9c37

Sonnet 5.5 now selects medium effort by default without an identified regression; the change is ready to merge after normal checks.

Architecture Summary

Architecture risk: 🔵 Low · up to f9c37

The change affects 1 system.

Changed systems: apps/server

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — apps/server (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in apps/server/src/provider/model-manifest.json: The manifest’s updatedAt value advances from 2026-10-07T19:00:00Z to 2026-10-07T21:00:00Z.
  • observed — Modified behavior in apps/server/src/provider/model-manifest.json: Adds the sonnet-5-5 profile with a Reasoning selector whose default is medium and whose options are low, medium, high, xhigh, max, and ultrathink. Ultrathink is prompt-injected and maps to null for Claude Code; the adapter sets a fixed context window of 1,000,000 tokens.
  • observed — Modified behavior in apps/server/src/provider/model-manifest.json: Claude Sonnet 5.5 now uses the sonnet-5-5 profile instead of sonnet-5.

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore (reviewers only)

❌ Failed checks (1 error)

Check name Status Explanation Resolution
Approvability ❌ Error Needs maintainer review under the product-default rule. In apps/server/src/provider/model-manifest.json, the new sonnet-5-5 profile marks medium as the effort default (lines 472-488), and `claud… A maintainer must review the product-default change in apps/server/src/provider/model-manifest.json before approval.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely identifies the main change: setting Claude Sonnet 5.5 to its shipped medium effort default.
Description check ✅ Passed The description includes all required sections, explains the problem and change, justifies the focused scope, links supporting evidence, and reports specific verification results and limitations.
Full details: Approvability

Explanation

Needs maintainer review under the product-default rule. In apps/server/src/provider/model-manifest.json, the new sonnet-5-5 profile marks medium as the effort default (lines 472-488), and claude-sonnet-5-5 now uses that profile (lines 686-692). The existing profile selected high, so users who do not choose an effort will receive a different effort. This changes a setting default.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

…onnet-5-5-default-effort

# Conflicts:
#	apps/server/src/provider/model-manifest.json

Copy link
Copy Markdown
Member

Note

This comment is posted by Julius' dot

Please attach the Claude Code 2.1.284 catalog excerpt or probe that establishes medium as Sonnet 5.5's default, and show the resulting T3 selection. #14152 describes high as the upstream default, while this PR and #14148 report medium. Resolving that discrepancy will establish whether this is a correction under the problem and scope rule.

…t-effort

# Conflicts:
#	apps/server/src/provider/model-manifest.json
@zachweyland

Copy link
Copy Markdown
Author

You asked for three things: the catalog value, what T3 actually selects, and why #14152 said high. Here they are.

Reproduce the catalog excerpt

The value lives in the installed binary's model table. On the version T3 gates Sonnet 5.5 behind (minVersion: 2.1.284), which is also what claude currently resolves to here:

claude --version   # 2.1.284 (Claude Code)
sha256sum ~/.local/share/claude/versions/2.1.284
# 5cd90aabd83f8a15136c35aa37bb1d92b348993573316643dc3fe4e04afbf88f
rg -ao '\{id:"claude-sonnet-5-5".{0,900}?\},\{id' ~/.local/share/claude/versions/2.1.284

The entry (byte offset 198719101) ends with:

context:{window:1e6,native_1m:!0,...},max_output_tokens:{default:128000,upper:128000},
pricing:"tier_2_10",capabilities:["effort","max_effort","xhigh_effort",...],
default_effort:"medium",image_limits:{...},advisor_rank:3

Not a stale-build artifact. Current stable 2.1.287 (linux-x64 binary, sha256 3920489a5109cff5786a1a392c25277408ff22bc796d5edb9c16a60e5a1718f0) has the same values: claude-sonnet-5-5 = medium, claude-sonnet-5 = high, claude-opus-5-5 = medium.

Why #14152 said high, and why that is only true of Sonnet 5

Same binary, the resolver for a model's default effort:

function ge(e){return Xa(Ue(e))?.default_effort??"high"}

Models that predate the field fall through to the hard-coded "high"; models that carry the field use it. Per-entry values, verbatim from the binary (identical on 2.1.284 and current stable 2.1.287):

model entry effective default
claude-sonnet-4-6 field absent high (fallback)
claude-sonnet-5 default_effort:"high" high
claude-sonnet-5-5 default_effort:"medium" medium
claude-opus-5 default_effort:"high" high
claude-opus-5-5 default_effort:"medium" medium

So "Sonnet 5.5 defaults to high like Sonnet 5" was true of every Sonnet through 5, and 2.1.284 moved 5.5 down to medium to match Opus 5.5. The discrepancy is between the old model and the new one, not between sources.

Why this matters for T3 specifically

T3 passes the resolved picker value to the CLI. getProviderOptionCurrentValue (packages/shared/src/model.ts:81) takes the profile's isDefault, ClaudeAdapter.ts:4852 runs it through resolveClaudeCatalogEffort, and the spawn appends --effort. An explicit flag outranks the CLI's own default, so under the current sonnet-5 profile a T3 session runs 5.5 at high while every other 2.1.284 client runs it at medium. The diff adds a sonnet-5-5 profile so the flag matches the shipped default.

Resulting T3 selection (live)

Three throwaway threads on a running server, started without touching the effort picker, read from the startSession span attributes (claude.query.model, claude.query.effort). Each returned a completed assistant turn, so these are live selections, not aborted spawns.

started (UTC) claude.query.model claude.query.effort expected
2026-10-01T18:28:50Z claude-opus-5-5 medium correct (own opus-5-5 profile)
2026-10-01T18:29:01Z claude-sonnet-5-5 high the bug (reuses sonnet-5 profile)
2026-10-01T18:29:16Z claude-sonnet-5 high correct (sonnet-5 profile)

claude-sonnet-5-5 selecting high is the bug, on the exact model, through the real codepath. claude-opus-5-5 selecting medium shows the mechanism already does the right thing when a model has its own medium profile. This PR gives sonnet-5-5 the profile it was missing, the same one Opus 5.5 already has, so the flag resolves to medium through that identical chain.

@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 1, 2026 — with ChatGPT Codex Connector
…ult-effort

# Conflicts:
#	apps/server/src/provider/model-manifest.json
…ult-effort

# Conflicts:
#	apps/server/src/provider/model-manifest.json

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge checks failed. Please resolve the failing checks before merging.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:M 30-99 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants