test: don't start a real stdio server in test_cli

[owtaylor]

Fixes the anyio 4.14.2 → 4.15.1 half of the dependency-canary failure in #588.

Root cause

tests/test__main__.py::test_cli called the unmocked cli(), which reached main() → mcp.run(transport="stdio") and started a real stdio MCP server against pytest's captured stdin/stdout.

mcp/server/stdio.py does:

# Purposely not using context managers for these, as we don't want to close
# standard process handles.
stdout = anyio.wrap_file(TextIOWrapper(sys.stdout.buffer, encoding="utf-8"))

That comment is wrong: TextIOWrapper.__del__ closes its underlying buffer. Under pytest sys.stdout.buffer is the capture plugin's temp file, so finalizing that wrapper closes pytest's global capture and every later test fails with ValueError: I/O operation on closed file. Reproducible with no MCP involved:

def test_a():
    w = TextIOWrapper(sys.stdout.buffer, encoding="utf-8"); del w; gc.collect()
def test_b():
    print("hi")   # ValueError: I/O operation on closed file

anyio only controls the timing. The wrapper is reachable from the stdin_reader/stdout_writer closures → tasks → root task, and anyio ≤ 4.14.2 cached the root task in a RunVar for the life of the process, so it was never finalized during a session. anyio 4.15.1 removed that cache (agronholm/anyio#1203), the graph becomes cyclic garbage, and an arbitrary later gc pass closes stdout. In the canary run the first casualty happened to be tests/test_audit.py, with 914 errors cascading from there.

The test wasn't earning its keep

• ~1.4s to start a real server.
• It only passed because main() raised an ExceptionGroup (pytest's fake stdin isn't readable), which cli() turned into sys.exit(1) — so pytest.raises(SystemExit) was asserting the crash path, not the intended behavior.
• Run with -s, against real terminal stdin, it hangs forever.

Change

Mock main() and assert cli() calls it. test_cli_fatal_error is added to cover the except Exception branch the old test was covering by accident, so __main__.py stays at 100% coverage.

Verification

With anyio==4.15.1, the full suite goes from 245 passed, 914 errors to green, across three pytest-randomly seeds. ruff check/format clean. The two remaining failures in tests/tools/test_services.py are pre-existing and environment-dependent (they query the real local systemd/sshd) — identical on the 4.14.2 baseline.

The TextIOWrapper finalizer is an upstream bug; I'll file it against modelcontextprotocol/python-sdk separately. It also bites any embedder that runs stdio_server() and then keeps working — they lose stdout.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
unittests	97.98% <100.00%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
tests/test__main__.py	100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[owtaylor]

The upstream bug seems to have been fixed as part of modelcontextprotocol/python-sdk#3117 - so no need to report it.

I considered here whether it would be good to actually start a stdio server as part of the test suite - sort of like a mini functional test. That would go well past what was tested here previously - which as is pointed was just the stdio server crashing because stdin is closed. Since we run the real functional tests as part of CI, I don't think it is necessary, though if we repeatedly see problems where we break server startup and it isn't caught until CI runs the functional tests, we could definitely add it later.

[panyamkeerthana]

non blocking but should we add autospec=True to this patch call to follow the testing guidance in AGENTS.md?

[owtaylor]

non blocking but should we add autospec=True to this patch call to follow the testing guidance in AGENTS.md?

When I looked at the (llm-created) patch with autospec in one branch but not the others, I thought about that too ... but I decided there was no benefit in autospec= in combination with a side_effect= that throws an exception ... the parameters passed in just don't matter.