Retry mechanism for ModelBehaviorError

[ProblemFactory]

I have extensively use this framework during the recent week. It performs pretty well, except on rare circumstances, the LLM will attempt to call a nonexistent tool, which crashed a whole 10-min agent run. Can you implement a retry mechanism allowing re-execute the errored LLM call to have a chance to recover from the it?

[rm-openai]

Hmm can you share some code? function_tool already catches errors and attempts to have the LLM run again. Would be useful to know where it crashed so that I can debug.

[ProblemFactory]

Hmm can you share some code? function_tool already catches errors and attempts to have the LLM run again. Would be useful to know where it crashed so that I can debug.

This is the exception occurred

Traceback (most recent call last):
  File "/app/python/deep_search/agents/executor.py", line 81, in run_main_agent
    async for event in resp.stream_events():
  File "/home/work/.local/lib/python3.11/site-packages/agents/result.py", line 186, in stream_events
    raise self._stored_exception
  File "/home/work/.local/lib/python3.11/site-packages/agents/run.py", line 537, in _run_streamed_impl
    turn_result = await cls._run_single_turn_streamed(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/work/.local/lib/python3.11/site-packages/agents/run.py", line 674, in _run_single_turn_streamed
    single_step_result = await cls._get_single_step_result_from_response(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/work/.local/lib/python3.11/site-packages/agents/run.py", line 755, in _get_single_step_result_from_response
    processed_response = RunImpl.process_model_response(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/work/.local/lib/python3.11/site-packages/agents/_run_impl.py", line 367, in process_model_response
    raise ModelBehaviorError(f"Tool {output.name} not found in agent {agent.name}")
agents.exceptions.ModelBehaviorError: Tool search_linkedin not found in agent DeepSearch

The agent is defined as

main_agent = Agent(
        "DeepSearch", 
        MAIN_AGENT_PROMPT, 
        tools=[
            search_google_news, 
            search_google,
            visit_page,
            get_linkedin_job_postings
        ]
    )

Run config

run_config = RunConfig(
        model=DEFAULT_MODEL, #bedrock claude-3.7-sonnet
        model_provider=OpenAIProvider(
            api_key=MODEL_API_KEY,
            base_url=MODEL_BASE_URL #litellm proxy URL
        )

I ran the agent with

Runner.run_streamed(
                main_agent, 
                self.user_query, 
                max_turns=50, 
                run_config=run_config
            )

[rm-openai]

Ah got it. I can add something to fix this.

[tim5ly]

@rm-openai hey! having the same question regarding retries and also want to know the proper way of using tenacity.retry with Runner.run_streamed (if possible).

Thanks!

[jackien1]

@rm-openai also curious about retries with Runner.run_streamed or what is the proper mechanism.

[leoxs22]

Also for rate-limit. I have a flow that might make 50 tool calls and I'm hitting the tokens per minute limit. I would expect it to continue with an exponential retry

[exiao]

Need this one also