Repository navigation
umbrella(guards): 85 enumerating files outside repo_tests/ are covered by no reach check — 5 verified instances #17400
Description
Activity
- added sub-issues
on Sep 24, 2026 Decision needed: what to do with
autobot-infrastructure/(32 enumerating files)This is the checklist item left open deliberately. I have now measured the population rather than
guessing at it, and the measurement says the directory should not be decided wholesale — it
holds three different kinds of file and only one of them is what this umbrella is about.What is actually in there
git ls-files 'autobot-infrastructure/*.py'filtered to files containing an enumerator
(tracked_paths,git ls-files,rglob(,os.walk(,.iterdir() gives 32, matching the
figure inguard_reach_meta_test.py's docstring. Splitting them by whether anything reads their
result as a verdict:Kind Count Evidence Tests of live pre-commit hooks 2 shared/scripts/hooks/pre-commit-no-tracked-symlink_test.py,…-no-gitignore-shadow_test.py. The hooks themselves are wired at.pre-commit-config.yaml:679,692, and the suites are collected by all three CI invocations —ci.yml:404,coverage.yml:107,test-durations.yml:101— which nameautobot-infrastructure/shared/scripts/hooksexplicitly since #14884.CI-gating 1 shared/scripts/phase_validation_system.py, referenced by.github/workflows/phase_validation.yml.One-shot utilities 29 migrations ( migrate_config_imports.py,migrate_consolidated_config.py), fixers (fix_bare_excepts.py,fix_whitespace.py), profilers (memory_profiler.py,comprehensive_code_profiler.py), analysis (analyze_duplicates.py,quick_microservice_analysis.py) and similar. None referenced by.pre-commit-config.yamlor any workflow.Why the split matters
The reach floor exists because a guard's green is read as a verdict about the tree. That
argument applies to the first two rows and does not apply to the third: nobody reads
analyze_duplicates.pyexiting 0 as "the tree is clean". Putting a floor on a migration script
buys nothing and creates 29 constants that decay — the exact failure #15928 and #17356 are about.Options
A — All 32 in scope, own wave. Consistent, no judgement call at the boundary. Cost: 29 floors
on scripts whose output is nobody's verdict, each needing re-measurement as the tree moves.B — Exclude the whole directory, reason recorded in the guard. Cheapest. Cost: leaves the 2
hook-test suites and the 1 CI-gating script uncovered, and those are genuinely guard-shaped —
they are the reason #14884 was filed.C — Split (recommended). Bring
autobot-infrastructure/shared/scripts/hooks/into scope in
the same wave asscripts/— already CI-collected, already guard-shaped, 2 files. Handle
phase_validation_system.pyas its own entry. Exclude the remaining 29 with the reason recorded
in the guard as a named decision ("one-shot utilities; exit status is not a verdict about the
tree"), so the exclusion is auditable rather than silent — the same discipline
_EXCLUDED_META_FLOORSalready applies inreach_floor_migration_test.py.Recommendation: C
It is the only option whose boundary is drawn on "is this output read as a verdict", which is
the property this umbrella is actually about, rather than on directory name. It also costs the
least real work — 3 files in scope, not 32.Stated limits
- I did not read the 29 utilities individually; they were classified by name plus the absence
of any pre-commit/workflow reference. If any of them is invoked from a script or a cron this
sweep did not search, that reclassifies it and the split should be redrawn. - I checked references in
.pre-commit-config.yamland.github/workflows/only. - I initially believed the 2 hook suites ran nowhere. That was wrong — bug(ci): 8 pre-commit hook tests cannot run — the harness invokes a shell builtin through subprocess #14884 found and fixed
exactly that, and the wiring is present today at the three call sites cited above. Recording the
correction because the wrong version of this table would have argued for option B.
cc @mrveiss — this is the one item on #17400 that should not be defaulted silently.
- I did not read the 29 utilities individually; they were classified by name plus the absence
- addedneeds-decisionBlocked on an owner decision; options and a recommendation are on the issueBlocked on an owner decision; options and a recommendation are on the issue
on Sep 24, 2026
Parent: #15826.
#15826's meta-guard was built —
repo_tests/guard_reach_meta_test.pyfails when a new tree-scanning guard arrives with no floor, carries a shrink-onlyGRANDFATHEREDlist mirrored against_GRANDFATHERED_BASELINE, and proves itself with a planted unfloored guard. That half is done and is not what this issue is about.What this issue is about is the boundary that guard states about itself, in its own docstring:
That is a correctly declared gap, not a hidden one — and it is still a gap. 85 enumerating files across four directories are covered by no reach check at all, and those directories hold the guards that actually run in pre-commit and CI.
Verified instances
Five guards outside
repo_tests/were read in full and confirmed to render "I could not look" as "I looked and it is clean". Each is a child of this issue.scripts/check_websockets_cap.pypipeline-scripts/check_no_literal_ttl_seconds.pymain()scripts/check_nosec_format.pyscripts/check_constraint_drift.pyrglobanchored to the process CWD, and the scan set is never validatedscripts/audit_api_wiring.pyThese are instances, not the disease. The disease is that nothing would have caught any of them, because no check examines that population.
What to build
repo_tests/toscripts/,tools/andpipeline-scripts/, grandfathering the existing unfloored guards in a shrink-only list that fails in both directions, exactly asGRANDFATHERED/_GRANDFATHERED_BASELINEalready do.autobot-infrastructure/(32 files) explicitly — in scope with its own wave, or excluded with the reason recorded in the guard, never left ambiguous.planted_unfloored_guard_test.pyalready proves therepo_tests/half. A meta-guard that cannot fail on the thing it exists for is the defect it is checking for.tools/lint/_scan_helpers.py:tracked_paths, which already raisesEmptyEnumerationrather than returning[]. Known call sites:scripts/check_nosec_format.py:107,scripts/check_script_exec_bits.py:99,scripts/check_python_file_size.py:176,scripts/check_shell_file_size.py:145. This is the cheapest systemic fix — it converts each empty-scan hole into a raise for free.Relationship to #14896
#14896 proposed
tracked_pathsfor the 35-checker harness and measured "33 of 35 checkers treat an empty scan as a clean scan" on 2026-08-22.tracked_pathsnow exists (tools/lint/_scan_helpers.py:200-204); the migration of call sites did not follow. #14896 owns the harness consolidation; this issue owns making coverage non-optional and the five verified instances. They should be sequenced, not merged — but they touch overlapping files, so one agent should hold both.Acceptance criteria
scripts/,tools/orpipeline-scripts/with no reach floor fails CI.Task tree
repo_tests/(this issue's own deliverable)scripts/(13 enumerating files)tools/(32)pipeline-scripts/(8)autobot-infrastructure/(32) — in scope with its own wave, or excluded with the reason recordedcheck_websockets_cap: a parse failure retires the capcheck_no_literal_ttl_seconds: its own reach counter is never calledcheck_nosec_format: no git rc check, no floor, silent on successcheck_constraint_drift: CWD-anchored enumeration, unvalidated scan setaudit_api_wiring: buckets never reconcile, frontend walk has no floor (blocked by audit_api_wiring.py is uneditable: function-length and size-ceiling gates interlock #17368)Sequencing: #17403 and #17404 share the
tracked_pathsmigration and should ride one PR. #17401 and #17402 are independent and small. #17405 cannot start until #17368 lifts the edit interlock. #17403 must be batched with #13554 (same file, different defect).Related, not duplicated
repo_tests/guard_reach_meta_test.py); this issue covers the population it explicitly does not reach.git ls-files, 5 of those unanchored, and 33 report clean on an empty scan #14896 — the 35-checker harness consolidation andtracked_paths. Overlapping files, so one owner; sequence rather than merge.Provenance
Found by an external-source comparison whose own conclusion was that our doctrine and our remedy plan are both ahead of the reference work; the useful output was this audit of our own tree, not anything adopted. Analysis:
docs/research/evidence-first-readiness-gating-toolkit.md.Numbers quoted from
guard_reach_meta_test.py's docstring and read in the tree atbcd7c8dbea; the79 of 133in #15826 was not re-measured in this pass.