test(sys-modules): scope six leaking stub owners the baseline never saw (#13450)

[mrveiss]

Thinking Path

The guard became a ratchet in #13442: an owner that leaks and is not on
repo_tests/sys_modules_leak_baseline.txt fails the run. Five owners were absent because
the baseline was generated from only the two invocation shapes CI uses, so every session
with a different path set reported pre-existing debt as a regression — the failure mode
#13398 exists to prevent.

The baseline only shrinks, so none of these could be listed. Each was fixed by binding its
stub to the window that actually needs it, per #13359's pattern
(autobot-backend/llc/tests/conftest.py): build the stub objects once so mock.patch's
getattr resolution keeps hitting the same object, install only the keys nothing else has
already provided, and remove exactly those again from a try/finally or through
monkeypatch.

Where the stub was only needed while tests run, monkeypatch.setitem is the whole fix —
its undo runs inside pytest_runtest_protocol, i.e. before the guard attributes the
node's delta, so the session sees no mutation at all. Only the two files with a
module-level import that genuinely needs the stub keep an install, and they wrap exactly
that import.

A sixth owner appeared the moment the fifth stopped leaking: setdefault is
first-writer-wins, so test_topology_routing_wiring.py had been invisible behind
test_learned_prompt_template_wiring.py for the same two keys. It is fixed the same way,
which is why this PR covers six files rather than five.

The sixth of the five listed owners — api/multimodal_response_test.py's four
transformers.* keys — is not fixed here. It is not a stub at all, and the analysis
and recommendation are below, left for a decision.

What Changed

Six test modules; no production code, no guard change, no baseline change.

File	Keys	Window it now uses
autobot-backend/llm_shared/structured_ops_test.py	utils, utils.semantic_chunker	monkeypatch.setitem + monkeypatch.setattr inside the one test that needs them
autobot-backend/tests/security/test_a2a_trust_gate.py	a2a.task_manager	autouse fixture; every api.a2a import is already inside a test body, so the import-time install was never needed
autobot-backend/tests/unit/chat_workflow/test_inline_judge.py	9 × langchain_core.* / langgraph.*	autouse fixture; graph.py is exec'd inside every test by _load_graph, so one test is the whole window
autobot-backend/tests/agents/test_learned_prompt_template_wiring.py	agents.agent_orchestration, …rl_router	try/finally around the module-level import
autobot-backend/tests/agents/test_topology_routing_wiring.py	same two keys	same try/finally
autobot-backend/tests/test_claude_adapter_wiring.py	anthropic	setdefault + delete-only-if-still-ours around the provider import

Two details that are load-bearing rather than tidiness:

• Stub objects are built once, at module level. Building a types.ModuleType mutates
  nothing, so it is safe outside the window; a fresh object per test would move the
  attributes mock.patch and the code under test resolve against (test-infra: llc/tests conftest leaks a sys.modules['agents'] stub — third conftest leak in two days #13337's trap).
• Only the keys this file installed are removed. _INSTALLED / the setdefault
  identity check mean a real, already-imported package is never displaced and never
  deleted — including the real anthropic, which CI installs.

Pre-existing test rot fixed in passing (test_inline_judge.py): two classes were
both named TestRouteAfterJudge, so the later one rebound the module attribute and the
four cases in the first were silently dropped from the suite. The second is renamed
TestRouteAfterJudgeRegenerationCap; collection goes 13 → 17 and all four recovered
cases pass.

Verification

Every "before" was captured by restoring the six files to origin/Dev_new_gui with
git restore --source=67d47406b and re-running the identical command.

Per owner — a session that actually exercises the escape:

Owner	Before	After
structured_ops_test.py	LEAK: … leaves 1 sys.modules key(s) … keys: utils.semantic_chunker	silent, 60 passed
test_a2a_trust_gate.py	LEAK: … keys: a2a.task_manager (escapes to tests/services)	silent, 30 passed
test_learned_prompt_template_wiring.py	LEAK: … keys: agents.agent_orchestration, agents.agent_orchestration.rl_router	silent, 35 passed
test_claude_adapter_wiring.py	LEAK: … keys: anthropic (escapes to autobot_shared)	silent, 55 passed
test_inline_judge.py	LEAK: … leaves 9 sys.modules key(s) … langchain_core, langgraph, …	silent, 1423 passed
test_topology_routing_wiring.py	LEAK: … keys: agents.agent_orchestration, agents.agent_orchestration.rl_router	silent, 9 passed

The session from the issue, verbatim, -n auto --dist loadscope:

before (files restored to base)

LEAK: autobot-backend/llm_shared/structured_ops_test.py leaves 1 sys.modules key(s) …
LEAK: autobot-backend/tests/agents/test_learned_prompt_template_wiring.py leaves 2 …
LEAK: autobot-backend/tests/security/test_a2a_trust_gate.py leaves 1 …
LEAK: autobot-backend/tests/test_claude_adapter_wiring.py leaves 1 …
4 file(s) above are NOT on repo_tests/sys_modules_leak_baseline.txt, so this run fails.
= 39 failed, 11551 passed, 330 skipped, 32 errors in 461.25s =

after

============================ sys.modules leak guard ============================
11 known leaking file(s), 113 sys.modules key(s), all on repo_tests/sys_modules_leak_baseline.txt — not a failure.
= 40 failed, 11554 passed, 330 skipped, 32 errors in 428.43s =

No LEAK: line, no FIXED: line — the baseline needs no edit.

Both CI shapes, collect-only:

pytest --collect-only -q -p no:randomly --continue-on-collection-errors \
  autobot-backend autobot_shared autobot-tts-worker repo_tests
  → 9 known leaking file(s), 71 keys, all on the baseline. 22706 tests collected.

pytest --collect-only -q -p no:randomly --continue-on-collection-errors autobot-slm-backend
  → 13 known leaking file(s), 19 keys, all on the baseline. 1566 tests collected.

Failure-set diff, before vs after, same command: +11554 −11551 passed (the four
recovered TestRouteAfterJudge cases, minus one) and exactly one failure that is not in
the before set:

FAILED autobot-backend/tests/orchestration/test_causal_error_recovery.py::TestRecoverySystemSmoke::test_recovery_plan_serialization

That one is pre-existing and independent of this PR — reproduced deterministically on
unmodified files, in a session that contains none of the six:

pytest autobot-backend/tests/orchestration/test_causal_error_recovery.py \
       autobot-backend/tests/agents/test_causal_reasoning.py -n 2 --dist loadscope
  → 1 failed, 35 passed          (-n 4 on the same two files: 36 passed)

Two baselined sys.modules jugglers restore overlapping keys at module teardown; when
the deal puts them on one worker the MagicMock stub is back before
test_recovery_plan_serialization's function-local re-import runs, and to_dict() dies
with TypeError: asdict() should be called on dataclass instances. Filed as #13457.
Adding four tests here is enough to shift the deal, which is all this PR did to it.

Lint on all six: black --check clean, flake8 clean, ruff check clean. The one
flake8 finding in test_topology_routing_wiring.py (F401 AgentTopology) is byte-for-byte
present on origin/Dev_new_gui and untouched here.

Model Used

Sonnet (claude-sonnet-4-6)

---

Left for a decision: api/multimodal_response_test.py and the four transformers.* keys

Not fixed here, deliberately. The full analysis is in the issue thread; the short version:

Where the entries come from. multimodal_response_test.py contains no sys.modules
statement, and neither does anything it reaches. grep across the whole repo finds zero
writes to any transformers* key. The chain is
multimodal_response_test.py:19 → api/multimodal.py:17 → ai_hardware_accelerator.py:44-50:

# Import transformers models for multi-modal embeddings
try:
    import librosa
    from transformers import CLIPModel, CLIPProcessor, Wav2Vec2Model, Wav2Vec2Processor

    MULTIMODAL_MODELS_AVAILABLE = True
except (ImportError, RuntimeError):
    MULTIMODAL_MODELS_AVAILABLE = False

The four entries are produced entirely inside the real library by its own lazy-module
machinery; the guard reports them as replaced because the keys already existed when the
from transformers import … above materialised them, and it attributes the delta to the
module whose import produced it.

It is a real dependency, installed in CI. requirements-ci/ai-ml.txt:4 pins
transformers==5.14.1. It is not in the local dev venv, which is why
_shadows_real_code filters it out locally and why this owner cannot be reproduced or
verified outside CI.

Nothing "relies on the entries" that AutoBot could stop relying on. They are the
library's own module registry, not a stand-in for anything. The only AutoBot state
involved is MULTIMODAL_MODELS_AVAILABLE, read at ai_hardware_accelerator.py:672.

Recommendation — treat it as a guard false positive, not a leak. Nothing is being
stubbed here, so there is no try/finally to add: the only production change available is
making the import lazy, which does not remove the delta, it relocates the blame to
whichever test first touches the multi-modal path, and it turns a module-level constant
read elsewhere into a call. That is churn with real blast radius and no fix in it.

The narrow alternative is a one-line refinement in _classify: a replaced entry whose
new object is a real, spec-carrying module the import system loaded is not a stub. It
cannot mask the three leaks the guard exists for — #13320 replaces sqlalchemy with a
MagicMock, #13324 replaces autobot_shared with a MagicMock, #13337 replaces agents
with a types.ModuleType, and all three read as __spec__ is None (Mock.__getattr__
raises on dunder names, so getattr(mock, "__spec__", None) is None).

It does have a cost, which is why it is a decision and not a unilateral change: at least
one baselined owner replaces with a real-loaded module —
llm_shared/npu_pipeline_routing_test.py builds its replacement with
module_from_spec(spec_from_file_location(...)), which carries a __spec__. Refining
_classify would silence it, and the shrink-only rule would then fail every run with
FIXED: … remove it from … until its line is deleted. So the change is "refine _classify
and delete the baseline lines it legitimately retires", which needs the owner's call on
how many lines that is.

Refs #13450
Refs #13361

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!