Skip to content

test(mcp): add runner-level coverage for upstream tool errors#2554

Open
OiPunk wants to merge 2 commits intoopenai:mainfrom
OiPunk:codex/openai-agents-879-mcp-runner-coverage
Open

test(mcp): add runner-level coverage for upstream tool errors#2554
OiPunk wants to merge 2 commits intoopenai:mainfrom
OiPunk:codex/openai-agents-879-mcp-runner-coverage

Conversation

@OiPunk
Copy link
Contributor

@OiPunk OiPunk commented Feb 26, 2026

Summary

This PR adds runner-level regression coverage for MCP upstream tool errors (issue #879).

Problem

Issue #879 reports that when an MCP upstream service returns an error (e.g., HTTP 422), the run can abort with an AgentsException instead of returning model-visible tool output.

Current code already implements graceful handling via failure_error_function in MCP tool wrappers, but there was no runner-level test that locks in this behavior end-to-end.

Changes

  • Added test_runner_handles_mcp_upstream_errors_as_tool_output:
    • Simulates an MCP server whose call_tool raises an upstream error.
    • Verifies Runner.run and Runner.run_streamed both continue and complete with the next model turn.
  • Added test_runner_raises_when_mcp_failure_error_function_disabled:
    • Verifies explicit mcp_config={"failure_error_function": None} preserves raising behavior.

Why this is useful

  • Prevents regressions on a real production concern: MCP upstream errors should not unexpectedly terminate runs under default configuration.
  • Clarifies behavior boundaries between default error handling and explicit opt-out.

Local validation

  • uv run --with pytest pytest -q tests/mcp/test_runner_calls_mcp.py
  • uv run --with ruff ruff check tests/mcp/test_runner_calls_mcp.py
  • uv run --with pytest pytest -q tests/mcp/test_mcp_util.py -k 'graceful_error_handling or failure_error_function'

Closes #879

@OiPunk
Copy link
Contributor Author

OiPunk commented Feb 26, 2026

Follow-up update:

  • The initial CI typecheck failure was caused by a mypy inference mismatch in the new test (RunResultStreaming vs RunResult variable reuse across branches).
  • I pushed c99c267a to split branch-local variables (streamed_result / non_streamed_result) without changing test behavior.

Revalidated locally:

  • uv run --with mypy mypy tests/mcp/test_runner_calls_mcp.py
  • uv run --with pytest pytest -q tests/mcp/test_runner_calls_mcp.py

Both pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP tools turn upstream HTTP errors into AgentsException, crashing the run

1 participant