Skip to content

fix(v3): avoid forced done tool use for Sonnet 5.5 - #3132

Open
shubh24 wants to merge 2 commits into
v3from
shubh24/sonnet-55-auto-done-v3
Open

shubh24 wants to merge 2 commits into
v3from
shubh24/sonnet-55-auto-done-v3

Conversation

@shubh24

@shubh24 shubh24 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add Claude Sonnet 5.5 to the models that reject forced tool use.
  • Let the final v3 agent done request use the existing auto tool choice for this model.
  • Add tests for model IDs with and without the anthropic/ prefix, a done call, and a text reply.
  • Add a patch changeset for @browserbasehq/stagehand.

Checks

  • ESM build passed.
  • The two related unit test files passed: 17 tests.
  • TypeScript, ESLint, Prettier, and changeset checks passed.
  • A small live Sonnet 5.5 test returned the forced-tool 400 with the original code. With this change, Sonnet called done and returned a valid result.

Scope

This fixes the final done request in the v3 DOM/hybrid agent path. A full browser run with prior thinking blocks has not been tested. Claude Sonnet 5.5 CUA support needs separate work.


Summary by cubic

Fixes the v3 agent's final done call returning a 400 error for Claude Sonnet 5.5 by using automatic tool choice for that model, and extends the same handling to Opus 5.5 and Fable 5.1.

  • Replaces the Fable-5-only provider check with an explicit set of model IDs that reject forced tool use, stripping the anthropic/ prefix.
  • Adds unit tests for model IDs with and without the prefix, a done call, and a plain-text reply.
  • Includes a patch changeset for @browserbasehq/stagehand.

Written for commit 2c8a209. Summary will update on new commits.

View guided diff Turn on auto-fix

@changeset-bot

changeset-bot Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2c8a209

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@browserbasehq/stagehand Patch
@browserbasehq/stagehand-evals Patch
@browserbasehq/stagehand-server-v3 Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@shubh24
shubh24 marked this pull request as ready for review October 7, 2026 19:23

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.
Architecture diagram
sequenceDiagram
    participant V3Agent as v3 Agent (DOM/Hybrid)
    participant DoneUtil as handleDoneToolCall.ts
    participant Opts as anthropicOptions.ts
    participant AI as AI SDK generateText
    participant Model as Claude Sonnet 5.5 API
    participant Tests as Unit Tests

    Note over V3Agent,Model: Final Done Request Flow

    V3Agent->>DoneUtil: Request final done tool call
    DoneUtil->>Opts: rejectsForcedToolUse(modelId)
    
    alt Model rejects forced tool use (Sonnet 5.5 or Fable 5)
        Opts->>Opts: Check MODELS_WITHOUT_FORCED_TOOL_USE set
        Opts-->>DoneUtil: True (model in set)
        DoneUtil->>AI: generateText with toolChoice: "auto"
        AI->>Model: Send request with automatic tool choice
    else Model accepts forced tool use (e.g. Opus)
        Opts-->>DoneUtil: False
        DoneUtil->>AI: generateText with toolChoice: "tool" (forced)
        AI->>Model: Send request with forced done tool
    end

    alt Model returns tool call
        Model-->>AI: tool-call: "done" with reasoning
        AI-->>DoneUtil: Done tool result
        DoneUtil-->>V3Agent: taskComplete: true
    else Model returns text reply
        Model-->>AI: text response (no tool call)
        AI-->>DoneUtil: Text content
        DoneUtil-->>V3Agent: taskComplete: false
    end

    Note over Tests: Unit Coverage
    Tests->>Opts: rejectsForcedToolUse("claude-sonnet-5-5")
    Opts-->>Tests: True
    Tests->>Opts: rejectsForcedToolUse("anthropic/claude-sonnet-5-5")
    Opts-->>Tests: True (prefix stripped)
    Tests->>Opts: rejectsForcedToolUse("claude-opus-4-8")
    Opts-->>Tests: False
    
    Tests->>DoneUtil: handleDoneToolCall with Sonnet mock
    DoneUtil->>AI: Verify toolChoice = auto
    AI-->>DoneUtil: Mock "done" tool call
    DoneUtil-->>Tests: taskComplete: true
    DoneUtil-->>Tests: reasoning preserved
    
    Tests->>DoneUtil: handleDoneToolCall with text response
    DoneUtil->>AI: Verify toolChoice = auto
    AI-->>DoneUtil: Mock text reply
    DoneUtil-->>Tests: taskComplete: false
Loading

View guided diff | Turn on auto-fix | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 4 files (changes from recent commits).

Confidence score: 5/5

  • agent-done-tool-choice.test.ts does not cover the auto-tool-choice path for claude-fable-5, despite that model being listed in MODELS_WITHOUT_FORCED_TOOL_USE. Add it to automaticToolChoiceModels.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/tests/unit/agent-done-tool-choice.test.ts">

<violation number="1" location="packages/core/tests/unit/agent-done-tool-choice.test.ts:6">
P3: `automaticToolChoiceModels` omits `claude-fable-5`, one of the four entries in `MODELS_WITHOUT_FORCED_TOOL_USE` (anthropicOptions.ts), so the `handleDoneToolCall` auto-tool-choice path for the base Fable model is never exercised here. Add it (or export the implementation set) so the test and source lists cannot drift.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment on lines +6 to +10
const automaticToolChoiceModels = [
"claude-sonnet-5-5",
"claude-opus-5-5",
"claude-fable-5-1",
].flatMap((modelId) => [modelId, `anthropic/${modelId}`]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: automaticToolChoiceModels omits claude-fable-5, one of the four entries in MODELS_WITHOUT_FORCED_TOOL_USE (anthropicOptions.ts), so the handleDoneToolCall auto-tool-choice path for the base Fable model is never exercised here. Add it (or export the implementation set) so the test and source lists cannot drift.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/core/tests/unit/agent-done-tool-choice.test.ts, line 6:

<comment>`automaticToolChoiceModels` omits `claude-fable-5`, one of the four entries in `MODELS_WITHOUT_FORCED_TOOL_USE` (anthropicOptions.ts), so the `handleDoneToolCall` auto-tool-choice path for the base Fable model is never exercised here. Add it (or export the implementation set) so the test and source lists cannot drift.</comment>

<file context>
@@ -3,7 +3,13 @@ import type { LanguageModelV2 } from "@ai-sdk/provider";
 import { handleDoneToolCall } from "../../lib/v3/agent/utils/handleDoneToolCall.js";
 
-function sonnetMock(modelId: string, response: "done" | "text") {
+const automaticToolChoiceModels = [
+  "claude-sonnet-5-5",
+  "claude-opus-5-5",
</file context>
Suggested change
const automaticToolChoiceModels = [
"claude-sonnet-5-5",
"claude-opus-5-5",
"claude-fable-5-1",
].flatMap((modelId) => [modelId, `anthropic/${modelId}`]);
const automaticToolChoiceModels = [
"claude-fable-5",
"claude-fable-5-1",
"claude-opus-5-5",
"claude-sonnet-5-5",
].flatMap((modelId) => [modelId, `anthropic/${modelId}`]);

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants