[BUG] multimodal not working with gemini 2.5 pro experimental model

[Abhimanyuponianitor]

Description

Added local file path handling in add image tool

Steps to Reproduce

This is the built in code in which i added local file path handling as well which was not there so it is working fine with open ai and most of the anthropic model as well but with claude 3.7 sonnet it gives none response error can you please help me with this what is the issue .....

from typing import Dict, Optional, Union

from pydantic import BaseModel, Field

from crewai.tools.base_tool import BaseTool
from crewai.utilities import I18N
import os
import base64

i18n = I18N()

class AddImageToolSchema(BaseModel):
image_url: str = Field(..., description="The URL or path of the image to add")
action: Optional[str] = Field(
default=None, description="Optional context or question about the image"
)

class AddImageTool(BaseTool):
"""Tool for adding images to the content"""

name: str = Field(default_factory=lambda: i18n.tools("add_image")["name"])  # type: ignore
description: str = Field(default_factory=lambda: i18n.tools("add_image")["description"])  # type: ignore
args_schema: type[BaseModel] = AddImageToolSchema

def _run(
    self,
    image_url: str,
    action: Optional[str] = None,
    **kwargs,
) -> dict:
    action = action or i18n.tools("add_image")["default_action"]  # type: ignore
    
    if os.path.exists(image_url):
        try:
            with open(image_url, "rb") as image_file:
                encoded_string = base64.b64encode(image_file.read()).decode("utf-8")
            image_url = f"data:image/jpeg;base64,{encoded_string}"
        except Exception as e:
            raise ValueError(f"Error encoding image: {e}")
    else:
        image_url=image_url

    content = [
        {"type": "text", "text": action},
        {
            "type": "image_url",
            "image_url": {
                "url": image_url,
            },
        },
    ]

    return {"role": "user", "content": content}

Expected behavior

Please help me with this
All models has different way of handling image input but i think that is not the issue because it is working with other anthropic models

Screenshots/Code snippets

from typing import Dict, Optional, Union

from pydantic import BaseModel, Field

from crewai.tools.base_tool import BaseTool
from crewai.utilities import I18N
import os
import base64

i18n = I18N()

class AddImageToolSchema(BaseModel):
image_url: str = Field(..., description="The URL or path of the image to add")
action: Optional[str] = Field(
default=None, description="Optional context or question about the image"
)

class AddImageTool(BaseTool):
"""Tool for adding images to the content"""

name: str = Field(default_factory=lambda: i18n.tools("add_image")["name"])  # type: ignore
description: str = Field(default_factory=lambda: i18n.tools("add_image")["description"])  # type: ignore
args_schema: type[BaseModel] = AddImageToolSchema

def _run(
    self,
    image_url: str,
    action: Optional[str] = None,
    **kwargs,
) -> dict:
    action = action or i18n.tools("add_image")["default_action"]  # type: ignore
    
    if os.path.exists(image_url):
        try:
            with open(image_url, "rb") as image_file:
                encoded_string = base64.b64encode(image_file.read()).decode("utf-8")
            image_url = f"data:image/jpeg;base64,{encoded_string}"
        except Exception as e:
            raise ValueError(f"Error encoding image: {e}")
    else:
        image_url=image_url

    content = [
        {"type": "text", "text": action},
        {
            "type": "image_url",
            "image_url": {
                "url": image_url,
            },
        },
    ]

    return {"role": "user", "content": content}

Operating System

Ubuntu 20.04

Python Version

3.10

crewAI Version

0.108.0

crewAI Tools Version

0.38.0

Virtual Environment

Venv

Evidence

Please help me with this what could be the issue

Possible Solution

it is not working for claude 3.7 sonnet

Additional context

.

[Vidit-Ostwal]

Hi @Abhimanyuponianitor, this should ideally work, in my opinion.

[lucasgomide]

@Abhimanyuponianitor could you share the error/logs?

[Abhimanyuponianitor]

hi @lucasgomide
Same issue i am facing when i used gemini-2.5 pro model for multimodal..... it should ideally work when we are are passing base64 encoded url but it is not working , for other gemini models it work, same issue i faced for claude 3. 7 sonnet but changing tool description and action it worked

first it gives none response from llm and throws this at the end

but when i tried using directly using lite llm but here it is not working

2 validation errors for LLMCallStartedEvent
messages.str
Input should be a valid string [type=string_type, input_value=[{'role': 'user', 'conten...his image in detail"}'}], input_type=list]
For further information visit https://errors.pydantic.dev/2.11/v/string_type
messages.list[dict[str,any]].1
Input should be a valid dictionary [type=dict_type, input_value='{'role': 'user', 'c...nal input question\n```', input_type=str]
For further information visit https://errors.pydantic.dev/2.11/v/dict_type

[lucasgomide]

@Abhimanyuponianitor I think this bug was fixed alredy.
Could you try upgrade CrewAI to the latest version/using the branch main instead

[Abhimanyuponianitor]

@lucasgomide in the latest version also i tried same error i am getting when using gemini 2.5 pro exp model

File "C:\Users\abhimanyu.ponia\AppData\Local\miniconda3\envs\test\Lib\site-packages\crewai\agents\crew_agent_executor.py", line 221, in _invoke_loop
raise e
File "C:\Users\abhimanyu.ponia\AppData\Local\miniconda3\envs\test\Lib\site-packages\crewai\agents\crew_agent_executor.py", line 155, in _invoke_loop
answer = get_llm_response(
^^^^^^^^^^^^^^^^^
File "C:\Users\abhimanyu.ponia\AppData\Local\miniconda3\envs\test\Lib\site-packages\crewai\utilities\agent_utils.py", line 163, in get_llm_response
raise ValueError("Invalid response from LLM call - None or empty.")
ValueError: Invalid response from LLM call - None or empty.

[lucasgomide]

It's not the same error, right?
Do you mind to enable LiteLLM debug and share with us their logs?

Just adding os.environ["LITELLM_LOG"] = "DEBUG"

[Abhimanyuponianitor]

crewai_litellm.log
here are logs for other models i am getting responses but for latest gemini 2.5 pro model getting none response in the old version also getting same none response issue only

i have modified the built in add image tool to handle local file path as well instead of passing image url passing base64 encoded string and it is working for all models and even other gemini models also as i mentioned earlier

it is not able to give response based on image url and base64 encoded url as well

@lucasgomide