[BUG] Gemini LLM Tool Calling Fails: `tools` parameter is `null` in API request with CrewAI v0.121.0 & LiteLLM v1.70.4

[geantendormi76]

Description

Hi CrewAI Team,

We are encountering a persistent issue where our Agent, configured with a Google Gemini LLM (via crewai.LLM), fails to make tool calls. The Agent gets stuck in the "Thinking..." phase, and our LLMCallStartedEvent listener consistently shows that the tools parameter in the API request to Gemini is null.

We have conducted extensive debugging and believe the issue lies in how CrewAI (v0.121.0) and LiteLLM (v1.70.4) handle the transmission of structured tool definitions to the Gemini API.

Below is a detailed breakdown of the bug, our environment, and our findings:

1. Describe the Bug

When using CrewAI v0.121.0 with Google Gemini LLM (e.g., gemini/gemini-2.0-flash via crewai.LLM), custom tools configured for an Agent are not being correctly passed to the underlying Gemini API. Specifically, when inspecting the LLMCallStartedEvent, the event.__dict__['tools'] field (which should contain the structured tool definitions for the LLM API call) is consistently null. This prevents the Gemini LLM from recognizing or utilizing its native function calling capabilities, causing the Agent to get stuck in the "Thinking..." phase without generating any Action: or Action Input:.

This issue persists despite extensive Prompt engineering, including:

• Ensuring all Pydantic args_schema fields for custom tools have non-null description and no default values.
• Adding model_config = ConfigDict(extra='forbid') to Pydantic args_schema models.
• Manually embedding a standard JSON Schema representation of args_schema within the tool.description attribute passed to the LLM in the text prompt.
• Providing clear, ReAct-style tool invocation examples within the Task description.

2. Environment and Library Versions

• CrewAI: 0.121.0
• crewai-tools: (Please specify your version, e.g., 0.x.y)
• LiteLLM: 1.70.4
• google-generativeai (SDK): 0.8.5
• Pydantic: v2.11.5
• Python: (Please specify your Python version, e.g., 3.10.x or 3.12.x)
• Operating System: WSL2 (Ubuntu)
• Network: Accessing Gemini API via a SOCKS5 proxy (socks5h://127.0.0.1:10808), with HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY environment variables correctly set. Necessary SOCKS support libraries (pysocks, httpx[socks]) are installed.
• LLM Configuration in CrewAI:

  from crewai import LLM
  gemini_llm = LLM(
      model="gemini/gemini-2.0-flash", 
      api_key=os.getenv("GEMINI_API_KEY"), 
      temperature=0.1, 
      max_tokens=2048
  )

3. Steps to Reproduce

• Define a custom tool inheriting from crewai.tools.BaseTool with a Pydantic args_schema.
• Configure an Agent with this custom tool and the Gemini LLM instance.
• Define a Task that explicitly guides the Agent to use this custom tool.
• Instantiate a Crew and kickoff() the task.
• Implement LLMCallStartedEvent and LLMCallCompletedEvent listeners to inspect the data sent to and received from the LLM.

4. Expected Behavior

The LLMCallStartedEvent.event.__dict__['tools'] field (or equivalent attribute holding the structured tool definitions passed to litellm.completion) should contain a list of tool definitions formatted兼容 Gemini API (e.g., a list of dictionaries, each with a function_declarations key). The Gemini LLM should then be able to recognize these tools and, when appropriate, return a response containing tool_calls to invoke the custom tool.

5. Actual Behavior & Logs

• The Agent gets stuck in the "Thinking..." phase indefinitely.
• The LLMCallStartedEvent.event.__dict__['tools'] field is consistently null.
• The system message in the prompt sent to the LLM (observed via LLMCallStartedEvent.messages) does correctly show the tool's name and a JSON Schema representation of its arguments (due to manual embedding in the tool's description attribute), but this text-based schema is not a substitute for the API-level tools parameter.
• Crucially, a direct test using litellm.completion() (outside of CrewAI) with the same Gemini model, API key, and a manually constructed tools parameter containing the Pydantic-derived JSON schema for the custom tool, successfully passes the tool definition to the Gemini API. LiteLLM's debug logs (Final returned optional params: {'tools': [{'function_declarations': [...]}]}) confirm this. The Gemini API, in this direct LiteLLM test, recognizes the tool (though it may still opt to respond with text asking for more clarity before calling it, which is acceptable LLM behavior). This indicates LiteLLM itself is capable of handling and transmitting the tool schema correctly when provided with it.

Log Snippet from LLMCallStartedEvent in CrewAI:

// event.__dict__
{
  "timestamp": "2025-05-24 01:32:09.566129",
  "type": "llm_call_started",
  // ... other fields ...
  "messages": [
    {
      "role": "system",
      "content": "You are 信息检索专家...\nTool Name: HybridRAGQueryTool\nTool Arguments (JSON Schema):\n```json\n{\n  \"additionalProperties\": false,\n  \"properties\": {\n    \"query\": {\n      \"description\": \"用户提出的原始查询文本。\",\n      \"title\": \"Query\",\n      \"type\": \"string\"\n    },\n    // ... other params ...\n  },\n  \"required\": [\"query\", ...],\n  \"title\": \"QueryRequest\",\n  \"type\": \"object\"\n}\n```\nTool Description: 【核心RAG工具】...\n\nIMPORTANT: Use the following format in your response..."
    },
    {
      "role": "user",
      "content": "\nCurrent Task: 用户查询是：'公司2024年第一季度的销售额是多少？'..."
    }
  ],
  "tools": null, // <--- THIS IS THE CORE PROBLEM
  "callbacks": [
    "<crewai.utilities.token_counter_callback.TokenCalcHandler object at 0x...>"
  ],
  "available_functions": null
}

6. Suspected Cause

The issue appears to stem from how CrewAI v0.121.0 (possibly in conjunction with LiteLLM v1.70.4) processes the Agent's tools list and prepares the tools parameter for the litellm.completion call when the target LLM is Gemini. Despite the google-generativeai package being installed and Pydantic schemas being well-defined (without default values and with ConfigDict(extra='forbid')), the structured tool definitions are not making it into the final API request's tools field.

This could be due to:

• An internal bug in CrewAI's LLM class or its tool handling logic specific to Gemini.
• An incorrect or missing step in the conversion of CrewAI BaseTool instances (and their Pydantic args_schema) into the format LiteLLM expects for Gemini's function_declarations.
• A misconfiguration or missing parameter in the crewai.LLM instantiation for Gemini that is necessary to enable full tool/function calling support via the API.

7. Request for Assistance

• Could you please investigate why the tools parameter is null in the LLMCallStartedEvent when using Gemini with CrewAI v0.121.0 and LiteLLM v1.70.4, even when a direct LiteLLM call with a manually constructed tools parameter works?
• Is there a known issue or a specific configuration required in CrewAI or its LLM class to ensure Pydantic args_schema from BaseTool instances are correctly translated into the tools (containing function_declarations) parameter for Gemini API calls via LiteLLM?
• Are there any working examples specifico to CrewAI v0.121.0+ and Gemini (gemini/gemini-2.0-flash or gemini-pro) that demonstrate successful native tool/function calling where the API request's tools field is correctly populated?

Thank you for your time and assistance with this critical issue.

Steps to Reproduce

Prerequisites:

1. Python environment with the following packages installed (versions reflect our testing environment):
   • crewai==0.121.0
   • crewai-tools==0.45.0 (请将 0.45.0 替换为您实际使用的 crewai-tools 版本，可以通过 pip show crewai-tools 查看)
   • litellm==1.70.4
   • google-generativeai==0.8.5
   • pydantic==2.11.5
   • python-dotenv
   • httpx[socks]
   • pysocks
2. A valid GEMINI_API_KEY environment variable set.
3. (Optional, for full reproduction of our setup) A SOCKS5 proxy running locally on 127.0.0.1:10808 that can access Google Gemini APIs. Environment variables HTTP_PROXY, HTTPS_PROXY, ALL_PROXY set to socks5h://127.0.0.1:10808. Note: The core tools: null issue seems independent of the proxy, as direct LiteLLM calls with tools work through the proxy.
4. An MCP proxy server (like mcpo) and a simple RAG MCP service. For a truly minimal example to demonstrate the tools: null issue, this MCP dependency can be mocked out in the tool's _run method to just return a string, as the problem occurs before tool execution.

Code to Reproduce (Minimal Example Focus):

• pydantic_models.py:

  from pydantic import BaseModel, Field, ConfigDict
  from typing import List, Dict, Any, Optional
  
  class QueryRequest(BaseModel):
      model_config = ConfigDict(extra='forbid')
      query: str = Field(description="用户提出的原始查询文本。")
      top_k_vector: int = Field(description="期望检索的向量搜索结果数量。")
      top_k_kg: int = Field(description="期望检索的知识图谱结果数量。")
      top_k_bm25: int = Field(description="期望检索的 BM25 关键词搜索结果数量。")

• custom_crewai_tools.py (Simplified for MRE):

  import json
  from typing import Type, ClassVar
  from pydantic import BaseModel
  from crewai.tools import BaseTool
  from pydantic_models import QueryRequest # Assuming pydantic_models.py is in the same directory
  
  class GeminiJSONSchemaBaseTool(BaseTool):
      _original_description: str = ""
      def __init__(self, **kwargs):
          if 'description' in kwargs: self._original_description = kwargs['description']
          super().__init__(**kwargs)
          if hasattr(self, 'args_schema') and self.args_schema: self._reformat_description_with_json_schema()
          elif self._original_description:
              self.description = f"Tool Name: {self.name}\nTool Arguments (JSON Schema):\n```json\nNo arguments required.\n```\nTool Description: {self._original_description}"
  
      def _reformat_description_with_json_schema(self):
          schema_str = "No arguments required."
          if self.args_schema:
              try:
                  if isinstance(self.args_schema, type) and issubclass(self.args_schema, BaseModel):
                      schema_dict = self.args_schema.model_json_schema(); schema_str = json.dumps(schema_dict, indent=2, ensure_ascii=False)
                  else: schema_str = str(self.args_schema)
              except Exception as e: print(f"Error generating JSON schema for {self.name}: {e}"); schema_str = str(self.args_schema)
          self.description = f"Tool Name: {self.name}\nTool Arguments (JSON Schema):\n```json\n{schema_str}\n```\nTool Description: {self._original_description}"
  
  class MinimalRAGTool(GeminiJSONSchemaBaseTool): # Changed from BaseMCPTool for MRE
      name: str = "MinimalRAGQueryTool"
      args_schema: Type[BaseModel] = QueryRequest
  
      def __init__(self, **kwargs):
          original_tool_description = "A minimal RAG tool for testing."
          super().__init__(name=self.name, description=original_tool_description, args_schema=self.args_schema, **kwargs)
  
      def _run(self, query: str, top_k_vector: int, top_k_kg: int, top_k_bm25: int) -> str:
          print(f"--- MinimalRAGTool Executed with query: {query} ---")
          return json.dumps({"status": "success", "final_answer": f"Mocked RAG result for '{query}'"})

• run_agent_mre.py (Minimal Reproducible Example):

  import os
  import json
  from crewai import Agent, Task, Crew, Process, LLM
  from crewai.utilities.events import base_event_listener, CrewAIEventsBus
  from crewai.utilities.events.llm_events import LLMCallStartedEvent, LLMCallCompletedEvent
  from custom_crewai_tools import MinimalRAGTool # Assuming custom_crewai_tools.py is in the same directory
  from dotenv import load_dotenv
  
  load_dotenv()
  
  GEMINI_MODEL_NAME = "gemini/gemini-2.0-flash"
  GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
  
  if not GEMINI_API_KEY:
      print("CRITICAL ERROR: GEMINI_API_KEY not set.")
      exit(1)
  
  try:
      llm_for_agent = LLM(model=GEMINI_MODEL_NAME, api_key=GEMINI_API_KEY, temperature=0.1, max_tokens=2048)
      print(f"Agent LLM configured: {GEMINI_MODEL_NAME}")
  except Exception as e:
      print(f"CRITICAL ERROR: Failed to configure LLM: {e}"); exit(1)
  
  minimal_rag_tool = MinimalRAGTool()
  
  researcher_agent = Agent(
      role='信息检索专家 (MRE)',
      goal='使用 MinimalRAGQueryTool 回答查询。',
      backstory='一个用于复现工具调用问题的AI助手。',
      llm=llm_for_agent,
      tools=[minimal_rag_tool],
      verbose=True,
      allow_delegation=False
  )
  
  research_task = Task(
      description="用户查询是：'{query}'。请使用 `MinimalRAGQueryTool` 工具进行搜索。严格按以下格式调用：\n```\nThought: 我需要用 MinimalRAGQueryTool 搜索 '{query}'。\nAction: MinimalRAGQueryTool\nAction Input: {{\"query\": \"{query}\", \"top_k_vector\": 3, \"top_k_kg\": 2, \"top_k_bm25\": 3}}\n```",
      expected_output="工具的JSON字符串输出。",
      agent=researcher_agent,
  )
  
  test_crew = Crew(agents=[researcher_agent], tasks=[research_task], process=Process.sequential, verbose=True)
  
  event_bus = CrewAIEventsBus()
  @event_bus.on(LLMCallStartedEvent)
  def on_llm_call_started(source: any, event: LLMCallStartedEvent):
      print("\n===== LLM Call Started (MRE) =====")
      print(f"Source: {type(source).__name__}")
      model_name = getattr(event, 'model', 'Unknown')
      if hasattr(event, '__dict__') and model_name == 'Unknown': event_dict = event.__dict__; model_name = event_dict.get('model_name', event_dict.get('model', 'Unknown'))
      print(f"Model: {model_name}")
      print("Messages: ", json.dumps(event.messages, indent=2, ensure_ascii=False) if isinstance(event.messages, list) else event.messages)
      print("Tools in API Call: ", json.dumps(event.tools, indent=2, ensure_ascii=False) if hasattr(event, 'tools') and event.tools else "None or Not Present")
      print("Event Dict Tools: ", json.dumps(event.__dict__.get('tools'), indent=2, default=str) if hasattr(event, '__dict__') else "No __dict__ or no tools key")
      print("================================\n")
  
  @event_bus.on(LLMCallCompletedEvent)
  def on_llm_call_completed(source: any, event: LLMCallCompletedEvent):
      print("\n===== LLM Call Completed (MRE) =====")
      # ... (similar logging as in your original run_agent.py) ...
      print("==================================\n")
      
  print("--- Starting MRE Crew ---")
  result = test_crew.kickoff(inputs={'query': 'Test query for MRE'})
  print("\n--- MRE Crew Finished ---")
  print("Final Result (MRE):", result.raw if hasattr(result, 'raw') else result)

Steps to run the MRE:

1. Save the above code blocks into pydantic_models.py, custom_crewai_tools.py (using MinimalRAGTool), and run_agent_mre.py respectively in the same directory.
2. Create a .env file with GEMINI_API_KEY="YOUR_KEY".
3. Ensure SOCKS5 proxy is set up in the terminal if needed.
4. Run python run_agent_mre.py.
5. Observe the console output.

Expected Result for MRE:
The LLMCallStartedEvent log should show the Tools in API Call (or Event Dict Tools) field populated with the schema for MinimalRAGQueryTool.

Actual Result for MRE:
The Tools in API Call (or Event Dict Tools) field in LLMCallStartedEvent is null. The agent gets stuck in "Thinking...".

Expected behavior

The LLMCallStartedEvent.event.__dict__['tools'] field (or an equivalent attribute holding the structured tool definitions passed to litellm.completion) should contain a list of tool definitions formatted for the Gemini API (e.g., a list of dictionaries, each with a function_declarations key).

The Gemini LLM should then be able to recognize these tools and, when appropriate, return a response containing tool_calls to invoke the custom tool, or at least a ReAct formatted text output indicating a tool call. The Agent should proceed to execute the tool.

Screenshots/Code snippets

{
  "role": "user",
  "content": "\nCurrent Task:

\u7528\u6237\u67e5\u8be2\u662f\uff1a'\u516c\u53f82024\u5e74\u7b2c\u4e00\u5b63\u5ea6\u7684\u9500\u552e\u989d
\u662f\u591a\u5c11\uff1f'\u3002\u4f60\u7684\u4efb\u52a1\u662f\u4f7f\u7528HybridRAGQueryTool\u5de5\u5177\u
5bf9\u7528\u6237\u67e5\u8be2\u8fdb\u884c\u6df7\u5408RAG\u641c\u7d22\u3002
\u4f60\u5fc5\u987b\u8c03\u7528\u8fd9\u4e2a\u5de5\u5177\u6765\u5b8c\u6210\u4efb\u52a1\u3002\u4ee5\u4e0b\u662
f\u8c03\u7528\u5de5\u5177\u7684\u7cbe\u786e\u793a\u4f8b\uff0c\u8bf7\u4e25\u683c\u9075\u5faa\u6b64\u683c\u5f
0f\uff1a\n\n\nThought: \u6211\u9700\u8981\u5bf9\u7528\u6237\u67e5\u8be2 '\u516c\u53f82024\u5e74\u7b2c\u4e00\u5b63\u5ea6\u7684\u9500\u552e\u989d\u662f\u591a\u5c11\uff1f' \u8fdb\u884c\u6df7\u5408RAG\u641c\u7d22\uff0c\u4ee5\u83b7\u53d6\u76f8\u5173\u4fe1\u606f\u3002\nAction: HybridRAGQueryTool\nAction Input: {{\"query\": \"\u516c\u53f82024\u5e74\u7b2c\u4e00\u5b63\u5ea6\u7684\u9500\u552e\u989d\u662f\u591a\u5c11\uff1f\", \"top_k_vector\": 5, \"top_k_kg\": 3, \"top_k_bm25\": 5}}\n\n\n\u8bf7\u52a1\u5fc5\u5c06 Action Input
\u4e25\u683c\u683c\u5f0f\u5316\u4e3a JSON
\u5b57\u7b26\u4e32\uff0c\u5e76\u786e\u4fdd\u5176\u4e2d\u7684\u53c2\u6570\u4e0e\u5de5\u5177\u7684
args_schema
\u5339\u914d\u3002\n\u4f60\u7684\u6700\u7ec8\u8f93\u51fa\uff08\u6307\u4f60\u8c03\u7528\u5de5\u5177\u540e\u7
684\u771f\u5b9e
Observation\uff09\u5c06\u662fRAG\u5de5\u5177\u7684\u539f\u59cb\u8f93\u51faJSON\u5b57\u7b26\u4e32\uff0c\u4e0
d\u8fdb\u884c\u4efb\u4f55\u4fee\u6539\u6216\u89e3\u91ca\u3002\u6ce8\u610f\uff1aHybridRAGQueryTool
\u7684\u8f93\u51fa\uff08\u771f\u5b9e\u7684
Observation\uff09\u4f1a\u662f\u4e00\u4e2aJSON\u5b57\u7b26\u4e32\uff0c\u5305\u542b 'status'
\u5b57\u6bb5\u3002\u5982\u679c 'status' \u662f 'error' \u6216
'clarification_needed'\uff0c\u8bf7\u5c06\u9519\u8bef\u4fe1\u606f\u6216\u6f84\u6e05\u8bf7\u6c42\u4f5c\u4e3a
u4f60\u7684\u601d\u8003\u7ed3\u679c\u7684\u4e00\u90e8\u5206\uff0c\u5e76\u505c\u6b62\u8fdb\u4e00\u6b65\u7684
\u56de\u7b54\u751f\u6210\uff0c\u4f8b\u5982\uff1a'Thought:
\u5de5\u5177\u8fd4\u56de\u4e86\u6f84\u6e05\u8bf7\u6c42\uff0c\u6211\u9700\u8981\u62a5\u544a\u7ed9\u7528\u623
7\u3002\nFinal Answer: \u7cfb\u7edf\u9700\u8981\u6f84\u6e05\uff1a[\u6f84\u6e05\u95ee\u9898]'\u3002\n\nThis
is the expected criteria for your final answer:
\u6df7\u5408RAG\u5de5\u5177\u7684\u539f\u59cb\u8f93\u51faJSON\u5b57\u7b26\u4e32\uff0c\u53ef\u80fd\u5305\u54
2b 'status', 'final_answer', 'clarification_question', \u6216 'error_message' \u7b49\u5b57\u6bb5\u3002\nyou
MUST return the actual complete content as the final answer, not a summary.\n\nBegin! This is VERY
important to you, use the tools available and give your best Final Answer, your job depends on
it!\n\nThought:"
}
],
"tools": null,
"callbacks": [
"<crewai.utilities.token_counter_callback.TokenCalcHandler object at 0x7ff336a374a0>"
],
"available_functions": null
}

============================================

🚀 Crew: crew
└── 📋 Task: 50be96fc-ef6f-4afc-94e9-b738123b35df
Status: Executing Task...
└── 🧠 Thinking...

Operating System

Ubuntu 22.04

Python Version

3.12

crewAI Version

0.121.0

crewAI Tools Version

0.45.0

Virtual Environment

Venv

Evidence

Key evidence is the LLMCallStartedEvent log showing event.dict['tools'] as null, consistently across multiple tests, even after ensuring Pydantic schemas are correctly defined without default values and with ConfigDict(extra='forbid').
A direct LiteLLM call (outside CrewAI, using litellm.completion) with a manually constructed tools parameter (containing function_declarations for the same tool schema) does show LiteLLM attempting to pass these tool definitions to the Gemini API (visible in LiteLLM's Final returned optional params). This strongly suggests the issue lies within CrewAI's preparation or passing of tool definitions to LiteLLM when targeting Gemini.

Please see the "Description" and "Steps to Reproduce" sections for detailed logs and MRE code.

Possible Solution

None

Additional context

We have been through extensive debugging, including:

• Verifying API key validity and network connectivity (SOCKS5 proxy is in use and confirmed working for direct LiteLLM calls to Gemini for text generation, and also for LiteLLM calls with tools when tools are manually constructed and passed to litellm.completion).
• Ensuring all Pydantic args_schema fields have non-null description, no default values, and use ConfigDict(extra='forbid').
• Manually embedding a standard JSON Schema representation of args_schema within the tool.description attribute (this successfully appears in the text prompt but does not solve the tools: null API parameter issue).
• Providing very explicit ReAct examples in Task descriptions.
• Updating all relevant libraries (crewai, litellm, google-generativeai, pydantic, httpx[socks], pysocks) to their latest or recent stable versions.
  The core issue remains that the structured tool definitions are not being passed at the API call level (tools: null), preventing Gemini from utilizing its native function calling.

[mouramax]

So, unfortunately, I couldn't reproduce the error the OP ran into with their code – it was throwing a bunch of different errors on my end. Anyway, from what I could gather, it looks like the main idea was to use a custom tool, nothing too complicated.

What kinda jumps out is the somewhat unorthodox way the OP declared the custom tool. That could be leading to some mixed signals in the JSON Schema, which in turn would mess with the tool calling. Same deal with the Task.description. It really should just lay out the task's goal from the user's angle, instead of trying to step on the toes of – or compete with – the detailed communication instructions that CrewAI handles internally (especially that whole internal back-and-forth during tool calling).

So, here’s a simplified and working version of what I think the OP was getting at, based on my best shot at understanding it:

from typing import Type
from pydantic import BaseModel, Field
from crewai.tools import BaseTool
from crewai import Agent, Task, Crew, LLM, Process
import json
import os

class QueryRequest(BaseModel):
    """Input schema for the MinimalRAGTool."""
    query: str = Field(
        ...,
        description="The required query string for the RAG tool."
    )
    top_k_vector: int = Field(
        ...,
        description="The required number of vector search results to fetch."
    )
    top_k_kg: int = Field(
        ...,
        description="The required number of knowledge graph results to fetch."
    )
    top_k_bm25: int = Field(
        ...,
        description="The required number of BM25 keyword search results to fetch."
    )

class MinimalRAGTool(BaseTool):
    name: str = "Minimal RAG Query Tool"
    description: str = "A minimal RAG tool for testing purposes."
    args_schema: Type[BaseModel] = QueryRequest

    def _run(self, query: str, top_k_vector: int, top_k_kg: int, top_k_bm25: int) -> str:
        print("\n--- MinimalRAGTool called ---")
        print(f"Query received: {query}")
        print(f"Top K Vector: {top_k_vector}")
        print(f"Top K KG: {top_k_kg}")
        print(f"Top K BM25: {top_k_bm25}")
    
        mock_result = {
            "status": "success",
            "final_answer": f"Mocked RAG result for query: '{query}'"
        }
        return json.dumps(mock_result)

os.environ["GEMINI_API_KEY"] = "<YOUR_KEY>"

minimal_rag_tool = MinimalRAGTool()

gemini_llm = LLM(
    model="gemini/gemini-2.0-flash",
    temperature=0.1,
    max_tokens=2048
)

researcher_agent = Agent(
    role="Information Retrieval Expert",
    goal="Answer user queries accurately and exclusively using the provided tools.",
    backstory=(
        "You are an advanced AI assistant specialized in information retrieval. "
        "Your expertise lies in efficiently utilizing tools to find the most "
        "relevant and precise answers to user queries."
    ),
    llm=gemini_llm,
    tools=[minimal_rag_tool],
    allow_delegation=False,
    verbose=True,
)

research_task = Task(
    description=(
        "You've received the following query from the user:\n\n'{query}'\n\n"
        "You should use the provided tools to process the query. If you need "
        "additional parameters for the tool, use the following default values: "
        "top_k_vector: 5, top_k_kg: 7, and top_k_bm25: 9. After utilizing the "
        "necessary tools, your output should be: 'My final answer is: ' "
        "followed by the exact, complete output from the tools used."
    ),
    expected_output=(
        "The phrase 'My final answer is: ' followed by the exact output from "
        "the tools used."
    ),
    agent=researcher_agent,
)

test_crew = Crew(
    agents=[researcher_agent],
    tasks=[research_task],
    process=Process.sequential,
    verbose=True
)

result = test_crew.kickoff(
    inputs={
        "query": "Test query for MRE"
    }
)

print(result.raw, end="\n\n")

[geantendormi76]

Hi @mouramax,

Thank you so much for your quick and insightful reply, and especially for providing a working simplified example! This is incredibly helpful.

I understand your points about our previous "unorthodox" tool declaration (manually embedding JSON schema in the description) and the overly detailed ReAct instructions in the Task.description. Your explanation that the Task.description should focus on the user's goal rather than competing with CrewAI's internal communication instructions makes a lot of sense.

The way your example handles optional tool parameters by suggesting default values within the Task.description (instead of using default=... in Pydantic Field definitions) seems like a key insight to avoid the Gemini API's sensitivity to the "default" keyword in schemas. This could very well be the solution to our persistent tools: null issue.

We will immediately refactor our code based on your simplified example, specifically:

1. Simplifying our custom tool declaration (HybridRAGTool) to directly inherit from BaseTool (via our BaseMCPTool) and removing the manual JSON schema embedding in its description.
2. Ensuring our Pydantic args_schema (QueryRequest) fields do not have default values (which we had already implemented based on earlier findings).
3. Significantly refactoring our research_task.description to align with your example, focusing on the task goal and providing default value suggestions for optional tool parameters via text.

We will test this revised approach thoroughly and report back with the results. Your guidance is highly appreciated!

Thanks again!

[JoJ3o]

I have a similar issue where tools are never passed as an argument to the LLM provider (in my case a local llama.cpp server).
I've scanned through the codebase and my current understanding is that CrewAI never utilizes the "tool calling" feature of LLMs. It always gets a "content" output from the LLM and parses the text themselves to "identify tool calls", instead of using the powerful "tool_calls" feature that is built into many llms nowadays.

The cause for my understanding is because of the following code:
The Agent class uses the CrewAgentExecutor class to invoke the LLM and passes the tools to the executor as seen in this function in the Agent class:

    def _execute_without_timeout(self, task_prompt: str, task: Task) -> str:
        """Execute a task without a timeout.

        Args:
            task_prompt: The prompt to send to the agent.
            task: The task being executed.

        Returns:
            The output of the agent.
        """
        return self.agent_executor.invoke(
            {
                "input": task_prompt,
                "tool_names": self.agent_executor.tools_names,
                "tools": self.agent_executor.tools_description,
                "ask_for_human_input": task.human_input,
            }
        )["output"]

The __init__() of CrewAgentExecutor also contains a tools parameters, so there are two ways for the tools to make it to the executor. The problem is that the Agent Executor calls get_llm_response() with neither the given tool_names/tools from the Agent function or the self.tools from the Executor:

                answer = get_llm_response(
                    llm=self.llm,
                    messages=self.messages,
                    callbacks=self.callbacks,
                    printer=self._printer,
                )

This in turn calls the actual llm implementation with llm.call() without ever using the tools parameter of the call function:

def get_llm_response(
    llm: Union[LLM, BaseLLM],
    messages: List[Dict[str, str]],
    callbacks: List[Any],
    printer: Printer,
) -> str:
    """Call the LLM and return the response, handling any invalid responses."""
    try:
        answer = llm.call(
            messages,
            callbacks=callbacks,
        )

Please correct me if my understanding is wrong, but if my understanding is correct, CrewAI is really missing out from a powerfull LLM feature by using their own parsing technique that is more prone to errors and hallucinations. Additionally I find that chaining tool calls within the same task, becomes more difficult with this approach.

I believe they did this, because many LLM didn't support tool calls natively a year ago, but that time has changed, so the framework should change with it.

[Vidit-Ostwal]

Please correct me if my understanding is wrong, but if my understanding is correct, CrewAI is really missing out from a powerfull LLM feature by using their own parsing technique that is more prone to errors and hallucinations. Additionally I find that chaining tool calls within the same task, becomes more difficult with this approach.

I think they way they have handled this is bit different.

According to my understanding, LLM can never call the tool itself, so what really happens when you give a tool to litellm .
They most probably generate a description around what available tool is, and how the different arguments are oriented.

This all together goes to the LLM to infer.
The LLM output some special token which are then utilised by host application to understand what tool calling needs to be done and how exactly.

Instead of passing this description making work to litellm, they are doing this by themselves.
This is all in my understanding, maybe the core maintainer can answer this better.

[mouramax]

Not a maintainer here, just a pretty ordinary user with opinions.

Yep, @Vidit-Ostwal, your understanding is perfectly fine. First and foremost, LLMs never call functions (or tools) themselves. It's ALWAYS a dialog, kind of like this:

1. "Hey, Mr. LLM, here's the menu of tools available today and their parameters for service."
2. "Thanks, Mr. Software! Let me see... (this is the Thought: part) Hey, I'd love to order the use of Tool_ABC (that's the Action: part) with params {'a': 18, 'b': 21} (the Action Input: part)."
3. "Great choice, sir, let me process it... Here's the result of that processing, sir: '18 times 21 equals 4895, trust me!' (This is the Observation: part)."

And then it all gets added to the message history, and the loop goes on until there's an eventual Final Answer:. It's a dialog, plain and simple text strings.

The success of this whole back-and-forth hinges on two things: first, CrewAI needs to lay out the menu of tools and the ordering instructions super clearly. Second, a lot depends on the LLM's ability to actually understand and follow those instructions effectively. More recently trained LLMs, and the ones with more parameters, are obviously much better at following instructions.

CrewAI tries to parse the LLM's request in a few different ways (using json.loads, ast.literal_eval, json5.loads, and finally json_repair). As a last resort in this attempt to figure out the LLM's tool request, there's the function_calling_llm attribute. This allows a different LLM, one that's particularly good at structured data extraction, to be used to pull out the tool call into the correct Pydantic schema. And of course, ultimately, this function_calling_llm can be something custom-built with all the bells and whistles that some edge case might require.

So basically, when it comes to tool usage (call it "function calling" or "tool use", either way you're using the right industry lingo), CrewAI as a framework is pretty opinionated on how things should run. But hey, that's exactly why we use frameworks, am I right? It's designed to cover a wide range of common scenarios and steer clear of most tricky edge cases. But even so, it still lets developers tweak and customize the system as much as they need to for their own specific, unique situations.

[geantendormi76]

感谢大家的关注和建议，我已经解决了这问题了。具体的解决方案是：

我们来总结一下是如何解决 tools: null（即 CustomGeminiLLM.call 方法接收到的 tools 参数为 None）这个核心问题的。

核心问题现象：

最初，尽管我们在 run_agent.py 中为 Agent 配置了工具（例如 HybridRAGQueryTool），但在运行时，我们自定义的 CustomGeminiLLM 的 call 方法接收到的 tools 参数始终是 None。这导致 LLM (Gemini) 无法得知有哪些可用工具，自然也无法生成工具调用指令。

问题根源分析（基于研究报告和实践）：

研究报告指出，crewai==0.120.1 版本的 Agent 在其内部处理机制上可能存在问题 。它未能成功地将其配置的 BaseTool 实例列表转换为 LLM (特别是遵循 OpenAI 函数调用/工具调用规范的 API，如 Gemini 通过 LiteLLM 访问时) 所期望的 List[dict] 格式，并将其传递给底层 LLM 封装器（即我们的 CustomGeminiLLM）的 call 方法 。简单来说，CrewAI 的 Agent 没有把“工具清单”正确地交给 CustomGeminiLLM。

解决方案（工作坊 - Workaround）：

由于我们无法直接修改 CrewAI 的内部源码，我们采取了一种“工作坊”策略，通过修改我们自己掌控的 CustomGeminiLLM 类来实现：

在 CustomGeminiLLM 初始化时接收并转换工具：

我们修改了 CustomGeminiLLM 的 init 方法，增加了一个 agent_tools: Optional[List[BaseTool]] 参数。
在实例化 CustomGeminiLLM 时（在 run_agent.py 中），我们将 Agent 需要使用的工具列表（researcher_tools = [hybrid_rag_tool]）传递给这个 agent_tools 参数。
init 方法内部调用了 _convert_crewai_tools_to_gemini_format 方法（这个方法我们确保它能正确将 BaseTool 实例列表转换为 LiteLLM 所需的 List[Dict[str, Any]] 格式）。
转换后的工具定义被存储在 CustomGeminiLLM 实例的一个新属性 _gemini_tools_cache 中。
在 CustomGeminiLLM.call 方法中使用缓存的工具：

我们修改了 CustomGeminiLLM.call 方法。
在该方法内部，我们检查传递进来的 tools 参数。如果它仍然是 None（表明 CrewAI Agent 依然没有成功传递工具），我们就使用之前在 init 中转换并缓存好的 self._gemini_tools_cache 作为备用。
这样，即使 CrewAI Agent “失职”，CustomGeminiLLM 也能确保将正确的工具定义传递给底层的 litellm.completion 调用。
效果：

通过上述修改，日志显示：

CALL - Tools received by CustomLLM.call: No (表明 CrewAI Agent 确实没传)
CALL - INFO: Tools argument was None, using cached tools. (表明我们的备用方案启动了)
CustomGeminiLLM.call - LiteLLM PARAMS (Preview): ... tools=Yes ... (表明工具最终被传入 LiteLLM)
CustomGeminiLLM.call - Detected tool_calls: ... (表明 Gemini 成功收到了工具定义并决定调用它)

[mouramax]

@geantendormi76, great to hear you got it working. Congrats, and wishing you great success with CrewAI! Feel free to close this issue.