[Fix] Tool format with Anthropic Beta APIs

[randombet]

Why are these changes needed?

User reported bug """new anthropic structured outputs we have to specify that we're using the beta API, which AG2 does. However, that beta API also changes the way we define tools, specifically through the parameter "api_style" which is locked to "tool" or "function" by AG2 (in register_for_llm). Then, we get errors of this kind: "tools.0: Input tag 'function' found using 'type' does not match any of the expected tags:
'bash_20250124', 'custom', 'text_editor_20250124',"""

Related issue number

Checks

Core LLM tests passed https://github.com/ag2ai/ag2/actions/runs/20291884861

• I've included any doc changes needed for https://docs.ag2.ai/. See https://docs.ag2.ai/latest/docs/contributor-guide/documentation/ to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[joggrbot]

📝 Documentation Analysis

All docs are up to date! 🎉

---

✅ Latest commit analyzed: a1dc1bd | Powered by Joggr

[randombet]

@claude Review this PR. Find bugs and check if it would break existing AG2 features.

[claude]

Claude finished @randombet's task in 1m 37s —— View job

---

Code Review Complete ✅

Summary

This PR fixes a bug where OpenAI-formatted tools ({"type": "function", "function": {...}}) were not properly converted when used with Anthropic's structured outputs beta API, causing 400 errors. The fix adds proper tool format conversion in the _prepare_anthropic_params method.

Analysis

Changes Made

autogen/oai/anthropic.py (7 lines added):

• Added tool format conversion for the tools parameter in _prepare_anthropic_params() method (lines 549-555)
• Mirrors existing conversion logic for functions parameter
• Converts OpenAI format to Anthropic format before API calls

Tests (502 lines added):

• Added comprehensive integration tests for tools + structured outputs
• Tests verify the fix works with groupchat, AutoPattern, and various agent configurations

Issues Found

1. Duplicate Tool Conversion Logic ⚠️

Location: autogen/oai/anthropic.py:549-555 and autogen/oai/anthropic.py:794-796

The tool conversion logic is now duplicated in two places:

1. _prepare_anthropic_params() - line 549-555 (NEW)
2. _create_standard() - line 794-796 (EXISTING)

Why this is a problem:

• When _create_standard() is called with tools parameter, it converts tools to functions (line 795-796)
• Then it calls _prepare_anthropic_params() which tries to convert tools again (line 549-555)
• But the tools key was already removed by _create_standard(), so the second conversion won't execute
• However, when _create_with_native_structured_output() or _create_with_json_mode() is called, they skip the first conversion and rely on _prepare_anthropic_params() to do the conversion

Current flow:

# Flow 1: _create_standard()
params = {"tools": [...]}
_create_standard(params)
  → converts tools to functions (line 795-796)
  → params now has "functions" key, no "tools" key
  → calls _prepare_anthropic_params(params)
    → handles "functions" conversion (line 545-548)
    → skips "tools" conversion (line 549-555) because key was removed

# Flow 2: _create_with_native_structured_output()
params = {"tools": [...]}
_create_with_native_structured_output(params)
  → calls _prepare_anthropic_params(params) directly
  → handles "tools" conversion (line 549-555) ✅

Verdict: The code works correctly but is confusing. Both conversion paths are needed because different callers have different expectations.

Recommendation: Add a comment explaining why both conversions exist:

# Note: _create_standard() converts tools→functions before calling this method,
# but _create_with_native_structured_output() and _create_with_json_mode() 
# rely on this method to handle tools conversion

2. Missing Test Coverage for Edge Cases ⚠️

The PR adds excellent integration tests but misses some edge cases:

Missing test scenarios:

• Tools with strict=True + response_format together (only tested in integration, not unit tests for the conversion logic)
• Mixed tools and functions parameters in the same request
• Empty tools list
• Tools with invalid/malformed structure

Impact: Medium - The main use cases are covered, but edge cases might fail silently

3. Codecov Shows 0% Patch Coverage ⚠️

From the codecov report:

Patch coverage is 0% with 4 lines in your changes missing coverage

Lines 549-555 are not covered by unit tests. The integration tests exercise the code path, but unit tests specifically targeting _prepare_anthropic_params() with the tools parameter are missing.

Recommendation: Add unit test:

def test_prepare_anthropic_params_with_tools(anthropic_client):
    """Test that tools parameter is converted correctly."""
    params = {
        "model": "claude-sonnet-4-5",
        "messages": [],
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "test_tool",
                    "description": "Test",
                    "parameters": {"type": "object", "properties": {}}
                }
            }
        ]
    }
    anthropic_messages = []
    result = anthropic_client._prepare_anthropic_params(params, anthropic_messages)
    
    # Verify tools were converted
    assert "tools" in result
    assert result["tools"][0]["name"] == "test_tool"
    assert "input_schema" in result["tools"][0]

Potential Breaking Changes

None identified. The changes are additive and maintain backward compatibility:

• Existing code using functions parameter continues to work
• Existing code using tools parameter now works correctly with structured outputs (was broken before)
• All existing tests pass

Security Considerations

✅ No security issues found

• No injection vulnerabilities
• No data leakage
• Tool conversion is type-safe

Code Quality

Strengths:

• ✅ Clear fix for a specific bug
• ✅ Comprehensive integration tests
• ✅ Good test coverage of various scenarios (groupchat, AutoPattern, etc.)
• ✅ Proper error handling
• ✅ Clear documentation in test docstrings

Areas for Improvement:

• ⚠️ Duplicate conversion logic (see Issue update all links #1)
• ⚠️ Missing unit test coverage for the specific lines changed
• ⚠️ Could add comment explaining the dual conversion paths

Recommendations

High Priority

1. Add unit test for _prepare_anthropic_params() with tools parameter to improve code coverage
2. Add clarifying comment explaining why tool conversion exists in both _prepare_anthropic_params() and _create_standard()

Medium Priority

3. Add edge case tests:
   • Mixed tools + functions parameters
   • Empty tools list
   • Malformed tool structures

Low Priority

4. Consider refactoring to eliminate duplicate conversion logic (not required for this PR, but good for future cleanup)

Verdict

✅ APPROVE with suggestions

The PR successfully fixes the bug where OpenAI tool format was causing 400 errors with Anthropic's structured outputs beta API. The fix is correct and well-tested with integration tests. The duplicate code is intentional (different call paths need it) but could use clarifying comments. The main concern is the 0% patch coverage, which should be addressed by adding unit tests.

The PR will NOT break existing AG2 features - all changes are additive and maintain backward compatibility.

---

[marklysze]

Thanks @randombet, this fixes the error when using both structured outputs and tools together

[codecov]

Codecov Report

❌ Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
autogen/oai/anthropic.py	0.00%	4 Missing ⚠️

Files with missing lines	Coverage Δ
autogen/oai/anthropic.py	19.56% <0.00%> (-58.61%)	⬇️

... and 210 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.