Skip to content

fix(repo-tests): close the last unguarded sweep and make the collection model walk the MRO (#14985, #14984) - #14999

Merged
mrveiss merged 6 commits into
Dev_new_guifrom
issue-14985-guard-followups
Aug 24, 2026
Merged

mrveiss merged 6 commits into
Dev_new_guifrom
issue-14985-guard-followups

Conversation

@mrveiss

@mrveiss mrveiss commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Three follow-ups left behind by PR #14980, all in the repo-root guard sweeps under repo_tests/, so they land together.

Thinking Path

#14985 — the third sweep with the same hole. PR #14980 fixed two guards that handed a bare ast.parse result straight to their detector; one unparseable file anywhere outside _SKIP took the module down with a raw SyntaxError naming neither the sweep nor the remedy. collected_modules_inert_on_import_test.py had the identical gap at two call sites and no harness directory in its _SKIP. The fix copies the shape #14980 landed, and deliberately does not degrade to a skip or a continue: a file the sweep cannot read is not a file the sweep has cleared, and silently dropping it is the under-reporting these guards exist to catch. While here, the issue asked for a second look at the other repo-root sweeps — infra_script_imports_resolve_test.py walks the repo root for requirements files with the harness directory missing from its skip set, and a requirements file belonging to another branch's worktree would have widened its _DECLARED set. Same under-reporting direction, so it is closed in the same commit.

#14984 — MRO-aware, not merely detectable. The task allowed either a real fix or turning the dormant blind spot into a loud failure. Both were done, because neither alone is sound. pytest asks cls.__init__ is not object.__init__, which walks the whole MRO; _has_init inspected the class's own body only. _init_state replaces it as a tri-state walk — own body, then bases defined in the same module, then bases followed into the module they were imported from, resolved against the roots pytest.ini puts on pythonpath (relative imports resolve against their own package; object short-circuits, being the very thing pytest compares against). The third state is the honest part: a base that still cannot be located is None, and None is never read as clean. _has_init treats it as __init__-present, so the class's methods count as offenders — the safe direction — and a dedicated assertion names the class and the unreachable base outright, so the failure is legible rather than merely safe. That is the "explicit, documented decision" the issue asked for instead of a silent assumption.

#14986 — decided, evidenced, and honestly left open. The issue framed this as a binary: run them in CI, or record them as deliberately local-only. Measurement made the answer neither. They should run — that skill is repo tooling whose findings are used in review, so the #13368 argument for tools/ applies unchanged — and running them proved it rather than settling it, because 6 of the 42 are red today. Recording them as "deliberately local-only" would have blessed six silent failures. But wiring them is blocked on structure, not policy, and that work is a different scope from hardening two guards. So the decision and its evidence are written into pytest.ini next to the existing #13368/#13879 notes, and #14986 stays open with Refs. Partial delivery does not close an issue.

What Changed

File Change
repo_tests/collected_modules_inert_on_import_test.py _parse_module helper raising a named AssertionError on SyntaxError; both sweep call sites routed through it; harness directory added to _SKIP; two regression tests; measured counts re-derived
repo_tests/infra_script_imports_resolve_test.py harness directory added to _SKIP_PARTS — its repo-root walk for requirements files had the same hole
repo_tests/test_methods_in_uncollected_classes_test.py _has_init replaced by the tri-state _init_state MRO walk plus _module_path / _imported_from / _module_classes / _parsed resolvers; _scan now also reports unresolvable bases; three new assertions; the live --collect-only shape module gained an inherited-__init__ class and a two-link chain; the #14984 gap comment removed
pytest.ini the #14986 decision recorded in the testpaths block, with the blockers enumerated

No production code is touched. Every change is inside the guards themselves or a config comment.

Verification

Local characterisation only — CI is authoritative. Interpreter: Python 3.10.12, pytest 9.0.2.

Ratchet numbers unmoved. Re-measured through _scan() before and after:

Measure Before After
autobot-backend offenders 77 77
autobot-frontend offenders 10 10
autobot-infrastructure offenders 19 19
interface stubs 2 2
modules swept (uncollected guard) 1978 1978
Test* classes with an unresolvable base — 0

The cross-module walk resolves both real subjects — TestFileServerConnectorAcceptance and TestWebCrawlerConnectorAcceptance reach ConnectorAcceptanceTest one package over through an absolute import — and finds it constructor-free. That is why no number moved, and it is pinned by an assertion so the resolver cannot quietly start returning "unknown" for everything and pass by treating them as offenders.

The inert-on-import guard's own counts were re-derived rather than trusted: 1978 modules / 0 fatal / 49 guarded, against 1973 / 2 / 46 recorded when it was written. The two fatal modules were fixed by #14980; the module count moved because the harness directory left the walk, taking the five never-collected modules of #14986 with it. Floors (1800, 30) unmoved. The stale comment is corrected with the diff shown.

Mutation evidence — every guarded line was broken and restored. No mutation is pushed.

Mutation Caught by
.claude removed from _SKIP test_the_harness_directory_is_out_of_the_sweep
_parse_module swallows SyntaxError into an empty module test_an_unparseable_swept_file_fails_loudly_instead_of_reading_as_clean
_has_init reverted to the own-body-only model 3 tests, including the live --collect-only comparison with pytest
unresolvable base returns False instead of None test_the_detector_follows_an_inherited_init_up_the_mro
cross-module resolution disabled 3 tests, including both cross-module assertions

The regression probes are written under an already-_SKIPped directory so no concurrently running sweep can see them, and the unlink is in a finally.

#14986 evidence. The four tracked suites collect 42 test functions. Measured: 36 passed, 6 failed. The failures are five behaviour mismatches in the verifiers' regexes and predicates (_extract_endpoint dropping a trailing path segment, _extract_test_subject returning an article instead of the identifier, ConfigVerifier.can_verify and _extract_config_value missing a port claim, CodeVerifier.can_verify rejecting its own category) plus one path expectation in the extractor suite — each needing its own judgement on whether the test or the implementation is authoritative. Blockers on collection, all confirmed: the directory cannot become an importable package path because neither path segment is a Python identifier, so the pkgutil.extend_path route of #13086 does not transfer; three subjects are hyphenated files loaded through spec_from_file_location; one suite imports its package bare and resolves only because an earlier-collected sibling put the directory on sys.path first; and verifiers/test_verifier.py matches python_files while being an implementation whose class carries __init__, so adding the path registers a new entry in the #14927 population. Adding the path without that work turns six silent failures into a red required gate.

What is left on #14986: move the suite or its subjects somewhere importable, drop the sys.path shims, keep verifiers/test_verifier.py out of collection, fix or adjudicate the 6 red tests, then add the path to testpaths and replace the note with a wiring entry.

Closes #14984, #14985
Refs #14986, #14927, #14980

Model Used

Claude Opus 5 (1M context) — claude-opus-5[1m]

… it cannot read (#14985)

The sweep is a denylist walk from the repo root, so it reaches files pytest's
testpaths allowlist never does. Two call sites handed the result of a bare
ast.parse straight to _exit_calls, so one unparseable file anywhere outside
_SKIP took the whole module down with a raw SyntaxError naming neither the
sweep nor the remedy.

Both now route through _parse_module, which raises an AssertionError naming the
path, the line, the syntax message and how to exclude a directory on purpose.
It does not degrade to a skip or a continue: a file the sweep cannot read is
not a file the sweep has cleared.

.claude joins _SKIP for the reason the two sibling guards already have it --
agent worktrees are checked out under it and hold other branches' work in
progress, which is not always parseable. The same gap in
infra_script_imports_resolve_test.py's repo-root walk is closed too: a
requirements file belonging to another branch would have widened _DECLARED.

Re-measured: 1978 modules (was 1983 before .claude left the walk; the five
dropped are the never-collected .claude/skills/claims-audit modules of #14986),
0 fatal, 49 guarded. Both floors (1800, 30) unmoved.
…init__ (#14984)

pytest asks cls.__init__ is not object.__init__, which walks the whole MRO.
_has_init inspected the class's own body only, so a Test* class inheriting a
constructor from a plain base was modelled as collected while pytest skips it
with a PytestCollectionWarning -- under-reporting, which is the exact direction
this guard exists to catch.

_init_state replaces it as a tri-state walk: own body, then bases in the same
module, then bases followed into the module they were imported from, resolved
against the roots pytest.ini puts on pythonpath. Relative imports resolve
against their own package. object is short-circuited, because it is the very
thing pytest compares against.

A base that still cannot be located is the third state, and it is never read as
clean: _has_init treats unknown as __init__-present, so the class's methods
count as offenders, and test_no_test_class_inherits_init_from_an_unresolvable_base
names the class and the base outright so the failure is legible rather than
merely safe. Measured zero on this branch.

The live --collect-only comparison now carries an inherited-__init__ class and
a two-link chain, so pytest itself confirms the rule rather than the model
asserting it alone.

Re-measured, unmoved: autobot-backend 77, autobot-frontend 10,
autobot-infrastructure 19, interface stubs 2, 1978 modules. The cross-module
walk resolves both real subjects (TestFileServerConnectorAcceptance and
TestWebCrawlerConnectorAcceptance reach ConnectorAcceptanceTest one package
over) and finds it constructor-free, which is why no number moved.
…ths (#14986)

42 test functions across four tracked modules under .claude/skills/claims-audit/
have never been collected, because testpaths is an allowlist and .claude is not
on it. The decision, with its evidence, now sits next to the #13368 and #13879
notes so the next sweep does not re-file it.

They SHOULD run: the skill is repo tooling whose findings are used in review, so
the #13368 argument for tools/ applies unchanged. Measuring them proved the
point rather than settling it -- 36 pass and 6 fail today, a regression nobody
could see. Wiring them is blocked on structure, not policy: the directory cannot
become an importable package path (neither .claude nor claims-audit is a Python
identifier, so the pkgutil.extend_path route of #13086 does not transfer), three
subjects are hyphenated files loaded through spec_from_file_location, one suite
imports its package bare and resolves only by collection order, and
verifiers/test_verifier.py matches python_files while being an implementation
whose class carries __init__.

Adding the path without that work turns 6 silent failures into a red required
gate, so this records the state and leaves #14986 open rather than closing it.
@github-actions

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant