@effect/ai-anthropic: Use Anthropic's native Structured Outputs instead of tool-call emulation

[davidgoli]

What is the problem this feature would solve?

Currently, @effect/ai-anthropic's AnthropicLanguageModel emulates structured output (generateObject) by creating a forced tool call — it wraps the schema as a tool's input_schema and sets tool_choice: { type: "tool", name } (see packages/ai/anthropic/src/AnthropicLanguageModel.ts lines ~336-348).

Anthropic has since released native Structured Outputs (now GA), which provides grammar-constrained JSON generation via the output_format parameter. This native approach:

• Guarantees JSON Schema compliance via constrained decoding at the token level
• Eliminates the tool-call workaround and its edge cases (e.g., the prompt ordering bug in @effect/ai Ordering bug in Prompt.fromResponseParts causes Anthropic to return 400 Status #5678)
• Supports Claude Sonnet 4.5 and Opus 4.1+
• Is the officially recommended approach per Anthropic's docs

The current tool-call emulation is less reliable, doesn't benefit from constrained decoding, and diverges from how @effect/ai-openai handles structured output (which already uses OpenAI's native response_format: { type: "json_schema" }).

What is the feature you are proposing to solve the problem?

Update AnthropicLanguageModel to use Anthropic's native output_format parameter when responseFormat.type === "json":

1. Map ProviderOptions.responseFormat to Anthropic's output_format: { type: "json_schema", json_schema: { name, schema } } instead of creating a forced tool call
2. Update the generated API types (Generated.ts) if they don't yet include the output_format field (may require regenerating from updated Anthropic API specs)
3. The LanguageModel.ProviderOptions.responseFormat already carries { type: "json", objectName, schema } — the mapping is straightforward
4. Reference: @effect/ai-openai's prepareResponseFormat() function already does the equivalent for OpenAI

Anthropic API docs: https://docs.anthropic.com/en/docs/build-with-claude/structured-outputs

What alternatives have you considered?

• Keep current tool-call emulation: Works but doesn't benefit from constrained decoding, more fragile with prompt ordering
• User-side workaround: Could bypass generateObject and call the Anthropic API directly with output_format, but this defeats the purpose of the provider abstraction

Related issues:

• From Discord: Dealing with Unsupported JSON Schema Keywords in OpenAI Structured Calls #4563 (JSON Schema keyword stripping) — Anthropic's constrained decoding may have its own keyword restrictions worth considering
• @effect/ai Ordering bug in Prompt.fromResponseParts causes Anthropic to return 400 Status #5678 (Prompt ordering bug) — partially caused by the tool-call emulation approach

[davidgoli]

Implementation Notes

Target branch

Per the contributing guide, this is a new non-breaking feature and should target next-minor.

Key files

File	Role
packages/ai/anthropic/src/AnthropicLanguageModel.ts	Main change — lines ~330-349 (tool-call emulation) and response parsing at ~745, ~785, ~884
packages/ai/anthropic/src/Generated.ts	Auto-generated from Anthropic's OpenAPI spec — must include output_format field on CreateMessageParams / BetaCreateMessageParams
packages/ai/anthropic/scripts/generate.sh	Code generator that pulls the spec from anthropics/anthropic-sdk-typescript
packages/ai/openai/src/OpenAiLanguageModel.ts	Reference — prepareResponseFormat() at line ~1408 already does the OpenAI equivalent

Prerequisites

Generated.ts is auto-generated via scripts/generate.sh, which pulls the OpenAPI spec from anthropics/anthropic-sdk-typescript. The output_format field must be present in that upstream spec before the types can be regenerated. If it's not there yet, the Anthropic SDK team would need to publish an updated spec first.

Implementation sketch

1. Regenerate Generated.ts (once output_format is in the upstream spec)

cd packages/ai/anthropic && bash scripts/generate.sh

This should add output_format as an optional field on both CreateMessageParams and BetaCreateMessageParams.

2. Update AnthropicLanguageModel.ts — request construction (~lines 330-349)

Current code wraps the schema as a forced tool call:

tools: responseFormat.type === "text"
  ? tools
  : [{
      name: responseFormat.objectName,
      description: Tool.getDescriptionFromSchemaAst(responseFormat.schema.ast) ?? "Respond with a JSON object",
      input_schema: Tool.getJsonSchemaFromSchemaAst(responseFormat.schema.ast) as any
    }],
tool_choice: responseFormat.type === "text"
  ? toolChoice
  : {
      type: "tool",
      name: responseFormat.objectName,
      disable_parallel_tool_use: true
    }

Replace with native output_format (modeled after @effect/ai-openai's prepareResponseFormat()):

tools,
tool_choice: toolChoice,
output_format: responseFormat.type === "text"
  ? undefined
  : {
      type: "json_schema",
      json_schema: {
        name: responseFormat.objectName,
        description: Tool.getDescriptionFromSchemaAst(responseFormat.schema.ast) ?? "Respond with a JSON object",
        schema: Tool.getJsonSchemaFromSchemaAst(responseFormat.schema.ast) as any
      }
    }

3. Update response parsing

With native structured output, the response arrives as a text content block containing valid JSON (not a tool_use block). The parsing logic at lines ~745, ~785, and ~884 currently extracts JSON from tool_use blocks when responseFormat.type === "json". These need to instead extract the text content and parse it as JSON.

4. Add changeset

---
"@effect/ai-anthropic": minor
---

Use Anthropic's native Structured Outputs (`output_format`) instead of tool-call emulation for `generateObject`

Notes

• Anthropic's constrained decoding may have its own JSON Schema keyword restrictions (related: From Discord: Dealing with Unsupported JSON Schema Keywords in OpenAI Structured Calls #4563) — worth testing with the same schemas that currently work via tool-call emulation
• This eliminates the prompt ordering issue described in @effect/ai Ordering bug in Prompt.fromResponseParts causes Anthropic to return 400 Status #5678 since tools are no longer involved in structured output
• The output_format approach supports Claude Sonnet 4.5+ and Opus 4.1+ — older models would need a fallback or clear error