- 
                Notifications
    You must be signed in to change notification settings 
- Fork 2.1k
fix(anthropic): allow images in user messages while filtering model turns #3286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. | 
| Summary of ChangesHello @Happy-ryan, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical bug in the Anthropic LLM integration that prevented the proper handling of image attachments in user messages. By introducing a role-specific check, the system now correctly allows images to be sent by users to Claude models, while maintaining the necessary filtering for model-generated content. This enhancement significantly improves the multimodal capabilities of the ADK for Claude models, ensuring alignment with the Claude API's specifications. Highlights
 Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either  
 Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a  Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
 | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request correctly fixes a bug where image parts were being filtered from user messages sent to Claude models. The logic change is concise and effective. The addition of comprehensive, parameterized unit tests is excellent, as it validates the fix across different message roles (user, model, assistant) and ensures the expected logging behavior. I have one suggestion to make the new warning log message more specific, which will improve its utility for debugging. Overall, this is a well-executed fix.
| if content.role != "user" and _is_image_part(part): | ||
| logger.warning("Image data is not supported in Claude for model turns.") | ||
| continue | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This logic correctly filters images for non-user roles. However, the accompanying warning message on line 146 is now slightly misleading as it only mentions "model turns", but this block also executes for the "assistant" role. To improve debugging clarity, consider making the log message more general or dynamic. Note that this would require updating the assertion in the new unit test.
| if content.role != "user" and _is_image_part(part): | |
| logger.warning("Image data is not supported in Claude for model turns.") | |
| continue | |
| if content.role != "user" and _is_image_part(part): | |
| logger.warning("Image data is not supported in Claude for role '%s'. Filtering image part.", content.role) | |
| continue | 
648d0da    to
    b1319fc      
    Compare
  
    
Link to Issue or Description of Change
Related issue: #3250
Problem:
The
content_to_message_paramfunction inanthropic_llm.pywas filtering out image parts from all message roles, including user messages. This was a bug because Claude API supports images in user turns but not in model/assistant turns. As a result, users couldn't send images to Claude models even though the API supports it.Solution:
Fixed the bug by adding a role check (
content.role != "user") before filtering image parts. Now:This ensures Claude models in ADK can properly handle image attachments in user messages while preventing errors from unsupported images in model turns.
Testing Plan
Unit Tests:
Test Results:
$ pytest tests/unittests/models/test_anthropic_llm.py::test_content_to_message_param_with_images -v ==================================================== test session starts ===================================================== platform darwin -- Python 3.11.14, pytest-8.4.2, pluggy-1.6.0 -- /Users/joonpyo.hong/Documents/GitHub/adk-python/.venv/bin/python3 cachedir: .pytest_cache rootdir: /Users/joonpyo.hong/Documents/GitHub/adk-python configfile: pyproject.toml plugins: mock-3.15.1, asyncio-1.2.0, anyio-4.11.0, xdist-3.8.0, langsmith-0.4.38 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function collected 3 items tests/unittests/models/test_anthropic_llm.py::test_content_to_message_param_with_images[user_role_with_text_and_image] PASSED [ 33%] tests/unittests/models/test_anthropic_llm.py::test_content_to_message_param_with_images[model_role_with_text_and_image] PASSED [ 66%] tests/unittests/models/test_anthropic_llm.py::test_content_to_message_param_with_images[assistant_role_with_text_and_image] PASSED [100%]Manual End-to-End (E2E) Tests:
Tested with Claude agent processing messages containing images:
Checklist
Additional context
This PR fixes issue #3250 point 2: "No support for embedded attachments in message part".
Bug Fix Details:
This enables proper Claude API image support in ADK while maintaining safety by preventing unsupported images in model responses.