You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Verify Claude Haiku 5.5 tool_reference / tool search against the live API, then enable #2914
Claude Haiku 5.5 (claude-haiku-5-5, #2892 / #2911) ships with client-side tool_reference loading and Anthropic native tool search turned off, because its support has not been checked against the live API. This issue tracks verifying both, then enabling them. It also tracks a separate decision: whether the hosted web-search default should move from Claude Haiku 4.5 to 5.5.
The same table lists Claude Haiku 4.5 with an identical row. Yet upstream #6474 (3d8f74357) found Haiku 4.5 rejects client-side tool_reference blocks, which is why every Haiku has been excluded since. So the table alone was already wrong once for Haiku.
The two switches fail differently:
Native tool search (packages/coding-agent/src/core/extensions/builtin/tool-search/native-support.ts, defaultSupportsToolSearch): a 400 disables injection for the session and falls back to local search. Cost: one failed request.
supportsToolReferences (packages/ai/src/utils/prompt-cache-ttl.ts, defaultSupportsToolReferences): drives message-anchored deferred loading (defer_loading plus a tool_reference in a tool_result). Its only automatic recovery matches Tool reference '…' not found in available tools (anthropic-messages.ts). Any other rejection message fails the turn.
The PR that adds Haiku 5.5 had no sanctioned Anthropic credential, so no probe could be run.
Acceptance criteria
One live probe on claude-haiku-5-5 for each mechanism, recording only the HTTP status and the error type/message, never headers or keys:
a request with a client tool_result that carries a tool_reference to a defer_loading tool;
a request that includes tool_search_tool_bm25_20251119 (or regex) with deferred tools, and lets the model search.
If both succeed: enable Haiku 5.5+ in both gates. The version-aware check is already written and was reverted in feat(ai): add Claude Haiku 5.5 to the catalog #2911: a Haiku floor of 5.5 in defaultSupportsToolReferences and defaultSupportsToolSearch. Add the cases back to 0000-anthropic-native-tool-search-contract.test.ts and anthropic-haiku-5-5.test.ts, and update packages/ai/src/types.ts and packages/coding-agent/docs/extensions.md.
If either fails: record the status and message here, keep that gate off, and (for tool_reference) decide whether the recovery matcher should also cover the new message.
Optional, same probe session: whether thinking: {type: "disabled"} is a 400 on Haiku 5.5. senpi currently hides off for it, unverified (generate-models.ts adaptive-only comment).
Web-search default model (websearch/search-model.ts)
DEFAULT_SEARCH_MODELS.anthropic stays claude-haiku-4-5. The hosted-search request (websearch/providers/anthropic.ts) sets no thinking, and Haiku 5.5 thinks adaptively by default at effort medium. Moving the default would add thinking latency and tokens to a latency-bound call, even though Haiku 5.5 is 10x cheaper per token than 4.5 (0.1 / 0.5 vs 1 / 5 per MTok).
To decide:
Measure p50/p95 latency and cost of one hosted search on Haiku 4.5 vs 5.5. For 5.5, test both as-is and with an explicit low effort, if the search request gains an effort control.
Summary
Claude Haiku 5.5 (
claude-haiku-5-5, #2892 / #2911) ships with client-sidetool_referenceloading and Anthropic native tool search turned off, because its support has not been checked against the live API. This issue tracks verifying both, then enabling them. It also tracks a separate decision: whether the hosted web-search default should move from Claude Haiku 4.5 to 5.5.Why off, when the docs list it
tool_search_tool_regex_20251119andtool_search_tool_bm25_20251119: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool#model-compatibility3d8f74357) found Haiku 4.5 rejects client-sidetool_referenceblocks, which is why every Haiku has been excluded since. So the table alone was already wrong once for Haiku.packages/coding-agent/src/core/extensions/builtin/tool-search/native-support.ts,defaultSupportsToolSearch): a 400 disables injection for the session and falls back to local search. Cost: one failed request.supportsToolReferences(packages/ai/src/utils/prompt-cache-ttl.ts,defaultSupportsToolReferences): drives message-anchored deferred loading (defer_loadingplus atool_referencein atool_result). Its only automatic recovery matchesTool reference '…' not found in available tools(anthropic-messages.ts). Any other rejection message fails the turn.Acceptance criteria
claude-haiku-5-5for each mechanism, recording only the HTTP status and the errortype/message, never headers or keys:tool_resultthat carries atool_referenceto adefer_loadingtool;tool_search_tool_bm25_20251119(or regex) with deferred tools, and lets the model search.defaultSupportsToolReferencesanddefaultSupportsToolSearch. Add the cases back to0000-anthropic-native-tool-search-contract.test.tsandanthropic-haiku-5-5.test.ts, and updatepackages/ai/src/types.tsandpackages/coding-agent/docs/extensions.md.tool_reference) decide whether the recovery matcher should also cover the new message.thinking: {type: "disabled"}is a 400 on Haiku 5.5. senpi currently hidesofffor it, unverified (generate-models.tsadaptive-only comment).Web-search default model (
websearch/search-model.ts)DEFAULT_SEARCH_MODELS.anthropicstaysclaude-haiku-4-5. The hosted-search request (websearch/providers/anthropic.ts) sets nothinking, and Haiku 5.5 thinks adaptively by default at effortmedium. Moving the default would add thinking latency and tokens to a latency-bound call, even though Haiku 5.5 is 10x cheaper per token than 4.5 (0.1 / 0.5 vs 1 / 5 per MTok).To decide:
Related
3d8f74357(Haiku 4.5 rejectstool_reference)