[Bugfix] Unknown tool called by agent will cause response 400

[BeautyyuYanli]

Bug Description

User:

Unknown tool name: 'count_nodes'. Available tools: search_by_text, query_zero_hop, query_one_hop, general_query, think

Fix the errors and try again.

Error Response:

openai.BadRequestError
Error code: 400 - {'error': {'code': 'invalid_parameter_error', 'param': None, 'message': '<400> InternalError.Algo.InvalidParameter: An assistant message with "tool_calls" must be followed by tool messages responding to each "tool_call_id". The following tool_call_ids did not have response messages: message[12].role', 'type': 'invalid_request_error'}, 'id': 

Reason

The message should be rendered to a tool_result instead of user message. Need tool_call_id for this RetryPromptPart.

May Fix: #562

[Wh1isper]

Just run into the same problem, I believe this could fix: #562

[BeautyyuYanli]

Just run into the same problem, I believe this could fix: #562

The issue can have multiple causes but most of them have been fixed. I believe this is one of the remaining pieces.

[Kludex]

Can you give me an MRE to reproduce the issue?

[BeautyyuYanli]

Can you give me an MRE to reproduce the issue?

def send_email(email: str, subject: str, body: str):
    return "OK"


agent = Agent(
    model=model,
    system_prompt="""
    You are a helpful assistant that can help me send emails.
    Firstly use `draft_email` to draft an email body.
    Then use `send_email` to send an email.
    """,
    tools=[send_email],
)

res = agent.run_sync(
    "I need to send an email to John Doe with the subject 'Meeting'"
)
print(res.data)

I made up the tool draft_email and prompt to use it first, the model will try to use the tool.

Using model qwen-2.5-72b

[Kludex]

Please give me an MRE I can copy/paste on my IDE and just run, imports are missing. If I need to do something to run this model, please give me instructions.

[BeautyyuYanli]

I created the mock model which always return a fake_tool call at first

from pydantic_ai.models.openai import OpenAIModel
from openai.types.chat.chat_completion import ChatCompletion, Choice
from openai.types.chat.chat_completion_message import ChatCompletionMessage
from openai.types.chat.chat_completion_message_tool_call import (
    ChatCompletionMessageToolCall,
    Function,
)
from pydantic_ai import Agent

import logfire

logfire.configure()
logfire.instrument_openai()
Agent.instrument_all()

cnt = 0


class MockOpenAIModel(OpenAIModel):

    async def _completions_create(  # type: ignore
        self,
        messages,
        stream,
        model_settings,
        model_request_parameters,
    ):
        global cnt
        if cnt == 0:
            cnt += 1
            return ChatCompletion(
                id="123",
                choices=[
                    Choice(
                        message=ChatCompletionMessage(
                            role="assistant",
                            tool_calls=[
                                ChatCompletionMessageToolCall(
                                    id="123",
                                    type="function",
                                    function=Function(name="fake_tool", arguments="{}"),
                                )
                            ],
                        ),
                        finish_reason="tool_calls",
                        index=0,
                    )
                ],
                created=123,
                model="gpt-4o",
                object="chat.completion",
            )
        else:
            return await super()._completions_create(
                messages,
                stream,
                model_settings,
                model_request_parameters,
            )


model = MockOpenAIModel(
    model_name="gpt-4o",
)


def send_email(email: str):
    return "Sent"


agent = Agent(
    model=model,
    tools=[send_email],
)


res = agent.run_sync("Send an email to example@example.com")
print(res.data)

[Kludex]

I need help to reproduce this issue without a mock.

I've run the model locally with: ollama run qwen2.5:72b, and I have this code:

import asyncio

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider


def send_email(email: str, subject: str, body: str):
    return 'OK'


model = OpenAIModel('qwen2.5:72b', provider=OpenAIProvider(base_url='http://127.0.0.1:11434/v1'))
agent = Agent(
    model=model,
    system_prompt="""
    You are a helpful assistant that can help me send emails.
    Firstly use `draft_email` tool to draft an email body.
    Then use `send_email` to send an email.
    """,
    tools=[send_email],
)


async def main():
    async with agent.iter(
        "I need to send an email to John Doe with the subject 'Meeting'. The content of the body should be created by you."
    ) as agent_run:
        async for node in agent_run:
            if Agent.is_model_request_node(node):
                print(node)
    print(agent_run.result.data)


asyncio.run(main())

I do see:

ModelRequestNode(request=ModelRequest(parts=[RetryPromptPart(content="Unknown tool name: 'draft_email'. Available tools: send_email", tool_name=None, tool_call_id='pyd_ai_3c1cf6bad3454a7db2bea67e9e9492c0', timestamp=datetime.datetime(2025, 3, 28, 15, 24, 18, 140805, tzinfo=datetime.timezone.utc), part_kind='retry-prompt')], kind='request'))

But I can't reproduce the 400...

[BeautyyuYanli]

I need help to reproduce this issue without a mock.

I've run the model locally with: ollama run qwen2.5:72b, and I have this code:

import asyncio

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider


def send_email(email: str, subject: str, body: str):
    return 'OK'


model = OpenAIModel('qwen2.5:72b', provider=OpenAIProvider(base_url='http://127.0.0.1:11434/v1'))
agent = Agent(
    model=model,
    system_prompt="""
    You are a helpful assistant that can help me send emails.
    Firstly use `draft_email` tool to draft an email body.
    Then use `send_email` to send an email.
    """,
    tools=[send_email],
)


async def main():
    async with agent.iter(
        "I need to send an email to John Doe with the subject 'Meeting'. The content of the body should be created by you."
    ) as agent_run:
        async for node in agent_run:
            if Agent.is_model_request_node(node):
                print(node)
    print(agent_run.result.data)


asyncio.run(main())

I do see:

ModelRequestNode(request=ModelRequest(parts=[RetryPromptPart(content="Unknown tool name: 'draft_email'. Available tools: send_email", tool_name=None, tool_call_id='pyd_ai_3c1cf6bad3454a7db2bea67e9e9492c0', timestamp=datetime.datetime(2025, 3, 28, 15, 24, 18, 140805, tzinfo=datetime.timezone.utc), part_kind='retry-prompt')], kind='request'))

But I can't reproduce the 400...

Send the message to API server will occur 400 on provider Azure and Aliyun, not sure about Ollama. (but Azure OpenAI model have fewer illusion so it is not always triggered)

[Kludex]

Which model from azure? qwen2.5:72b?

[BeautyyuYanli]

Which model from azure? qwen2.5:72b?

Azure OpenAI model have fewer illusion so it is not always triggered, that's why I write the mock model...

[BeautyyuYanli]

import asyncio
from pydantic_ai.models.openai import OpenAIModel
from openai.types.chat.chat_completion import ChatCompletion, Choice
from openai.types.chat.chat_completion_message import ChatCompletionMessage
from openai.types.chat.chat_completion_message_tool_call import (
    ChatCompletionMessageToolCall,
    Function,
)
from pydantic_ai import Agent
from pydantic_ai.providers.openai import OpenAIProvider
import logfire

logfire.configure()
logfire.instrument_openai()
Agent.instrument_all()

model = OpenAIModel(model_name="gpt-4o-mini")


def asjhdgajhgd(jsdkfhskjdf: str):
    return "This is a dummy tool, not for sending emails. try tool `send_email` with parameter `email` instead."


agent = Agent(
    model=model,
    system_prompt="""
    You are a helpful assistant.
    You must call the tool `send_email` to send an email.
    The tool name `asjhdgajhgd` is wrong, it should be `send_email`.
    """,
    tools=[asjhdgajhgd],
)


async def main():
    async with agent.iter("Send an email to alice@google.com") as agent_run:
        async for node in agent_run:
            if Agent.is_model_request_node(node):
                print(node)
    print(agent_run.result.data)


asyncio.run(main())

I make the tool more dirty so that gpt-4o-mini on Azure will call the fake send_email tool

[Wh1isper]

import asyncio
from pydantic_ai.models.openai import OpenAIModel
from openai.types.chat.chat_completion import ChatCompletion, Choice
from openai.types.chat.chat_completion_message import ChatCompletionMessage
from openai.types.chat.chat_completion_message_tool_call import (
    ChatCompletionMessageToolCall,
    Function,
)
from pydantic_ai import Agent
from pydantic_ai.providers.openai import OpenAIProvider
import logfire

logfire.configure()
logfire.instrument_openai()
Agent.instrument_all()

model = OpenAIModel(model_name="gpt-4o-mini")


def asjhdgajhgd(jsdkfhskjdf: str):
    return "This is a dummy tool, not for sending emails. try tool `send_email` with parameter `email` instead."


agent = Agent(
    model=model,
    system_prompt="""
    You are a helpful assistant.
    You must call the tool `send_email` to send an email.
    The tool name `asjhdgajhgd` is wrong, it should be `send_email`.
    """,
    tools=[asjhdgajhgd],
)


async def main():
    async with agent.iter("Send an email to alice@google.com") as agent_run:
        async for node in agent_run:
            if Agent.is_model_request_node(node):
                print(node)
    print(agent_run.result.data)


asyncio.run(main())

I make the tool more dirty so that gpt-4o-mini on Azure will call the fake send_email tool

Providing some of my previous context as too many code changes I can't get back to this anymore ......

In my case, I wanted llm to do the planning for the tool calls, so I provided a series of tools in the system prompt, while including the context in which it used to call the tools, without actually giving llm those tools.

This time llm could potentially call these tools that are not in the definition.

[Kludex]

Thanks for keeping with me. :)

[BeautyyuYanli]

Thanks for keeping with me. :)

Thx for review 🙏