fix(audit-script): filter test_*.py + detect router_registry callers (#7128, #7109) by mrveiss · Pull Request #7168 · mrveiss/AutoBot-AI

mrveiss · 2026-05-07T13:43:05Z

Summary

The 2026-05-06 `audit-unwired-trackers.py` run on the live codebase produced 252 findings (filed as evidence on #6872). Two heuristic gaps made them all false positives:

Pytest `test_*.py` prefix-style modules weren't filtered (fix(audit-script): audit-unwired-trackers.py reports test_*.py files as unwired (#7109 sibling, #6872 follow-up) #7128). The existing `TEST_PATH_FRAGMENTS` only matched `_test.py` suffix, `.test.ts`, `/tests/`, `/tests/`. Files like `agent_loop/test_loop_repetition.py` leaked through.
Router-registry-wired modules were reported as orphan (discovery(audit-script): audit-unwired-trackers.py misses feature_routers.py registry callers (#6872 follow-up) #7109). Modules registered via `autobot-backend/initialization/router_registry/*.py` tuples (e.g. `("api.captcha", "", [...], "captcha")`) look caller-less to the import-grep heuristic but are live FastAPI routers.

This PR closes both gaps in one place since they share the same file.

Changes

`is_test_path()` now also returns True when `path.name.startswith("test_")` and the file is `.py`. We use a basename check (not substring) because pytest's own `tmp_path` fixture lives under `test_/` directories and would otherwise cause the audit walker to skip its own test setup.
`load_router_registry_modules()` parses every `router_registry/*.py` for `("module.path", ...)` tuples and returns the set of registered dotted paths. Found 144 modules registered this way in the current codebase.
`derive_module_path()` converts a file path like `autobot-backend/api/captcha.py` to the dotted name `api.captcha`.
`scan()` skips files whose derived module path is in the registry set.

Result

Re-running the audit on Dev_new_gui:

```bash
$ python3 pipeline-scripts/audit-unwired-trackers.py
✅ No unwired-tracker findings.
```

252 → 0. All prior findings were either pytest test files or registry-wired routers.

Test plan

51/51 audit-script unit tests pass (added 11 new cases):
- 4 new `is_test_path` cases (prefix-style + edge cases)
- 5 `derive_module_path` parametrized cases
- `load_router_registry_modules`: extracts paths + handles missing dir
- `scan`: end-to-end registry-skip + sanity check for true orphans
Audit dry-run on full codebase: 0 findings (down from 252)
Spot check: `api.captcha` (was flagged FP, now correctly identified as registered) — `load_router_registry_modules()` returns set containing `api.captcha`

Closes #7128
Closes #7109

🤖 Generated with Claude Code

…7128, #7109) The 2026-05-06 audit run produced 252 findings, ~all false positives. Two heuristic gaps caused them: 1. Pytest's test_*.py prefix-style modules were flagged as unwired production code. The existing TEST_PATH_FRAGMENTS only covered the _test.py / .test.ts suffix style, /tests/ + /__tests__/ directories. Files like agent_loop/test_loop_repetition.py slipped through. 2. Modules wired via the autobot-backend/initialization/router_registry/ tuple-based runtime lookup (e.g. ("api.captcha", "", [...], "captcha")) look caller-less to the import-grep heuristic. captcha, enhanced_memory, etc. — all live FastAPI routers — were reported as orphaned. Fixes: - is_test_path() now also returns True when path.name starts with "test_" AND has .py extension. Basename check (not substring) — pytest's tmp_path embeds `test_<id>/` directory names that would otherwise short-circuit the audit walker on its own test fixtures. - New load_router_registry_modules() parses every router_registry/*.py for ("module.path", ...) tuples and returns the set of registered dotted paths. Found 144 modules registered this way in the current codebase. - New derive_module_path() converts a file path like autobot-backend/api/captcha.py to the dotted name api.captcha. - scan() now skips files whose derived module path is in the registry set. Result: re-running the audit goes from 252 findings → 0. All previously reported "unwired" modules either (a) had test_*.py basenames or (b) were runtime-registered routers. Tests: - 4 new is_test_path cases (test_*.py prefix style + edge cases) - 5 new derive_module_path parametrized cases - load_router_registry_modules: extracts dotted paths + handles missing dir - scan: end-to-end registry-skip + sanity test for true orphans - 51 / 51 audit-script tests pass Closes #7128 Closes #7109

github-actions · 2026-05-07T14:09:07Z

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!

…ve path check (#7128b) #7168 (originally tagged #7128) closed with a stale "0 findings" proof — the script was being run from inside a worktree where every file's absolute path contains `/.worktrees/`, hitting the SKIP_PATH_FRAGMENTS filter and silently dropping ALL findings. Real count post-#7168 was ~165, not 0. Two bugs fixed here: 1. **Worktree-relative skip check.** `should_skip_path()` matched against `str(path)` (absolute), so any audit run with REPO_ROOT under `.worktrees/` filtered out everything. Now matches against the path relative to REPO_ROOT, so `.worktrees/` segments in REPO_ROOT itself are no longer false-positives, and nested worktree clones inside the repo are still excluded as before. 2. **Entry-point script filter.** Runner conventions like `intelligence/demos/run_intelligent_agent.py` (created by #7127) cite their tracker in the docstring and have 0 callers by design — they're designed to be invoked from CLI, not imported. New helper `is_entry_point_script(path)` skips files whose name starts with `run_` OR whose path contains a `demos/`, `scripts/`, or `bin/` directory segment. Test deltas: - `test_should_skip_path` parametrize cases switched from absolute paths to REPO_ROOT-relative inputs (matches the contract; prevents the same bug recurring). - New `test_should_skip_path_handles_running_from_worktree` regression test pins the fix. - New `test_is_entry_point_script` (10 cases) and `test_scan_skips_entry_point_runners` (end-to-end) cover the entry-point filter. 63/63 audit-script tests pass. Audit count dropped from 165 → 110 in local re-run; remaining 110 are real-or-different-FP-class candidates (lazy imports, dataclass field refs, importlib string lookups — separate follow-ups out of scope). Refs #7128 #7109 #6872

…ve path check (#7128b) (#7244) #7168 (originally tagged #7128) closed with a stale "0 findings" proof — the script was being run from inside a worktree where every file's absolute path contains `/.worktrees/`, hitting the SKIP_PATH_FRAGMENTS filter and silently dropping ALL findings. Real count post-#7168 was ~165, not 0. Two bugs fixed here: 1. **Worktree-relative skip check.** `should_skip_path()` matched against `str(path)` (absolute), so any audit run with REPO_ROOT under `.worktrees/` filtered out everything. Now matches against the path relative to REPO_ROOT, so `.worktrees/` segments in REPO_ROOT itself are no longer false-positives, and nested worktree clones inside the repo are still excluded as before. 2. **Entry-point script filter.** Runner conventions like `intelligence/demos/run_intelligent_agent.py` (created by #7127) cite their tracker in the docstring and have 0 callers by design — they're designed to be invoked from CLI, not imported. New helper `is_entry_point_script(path)` skips files whose name starts with `run_` OR whose path contains a `demos/`, `scripts/`, or `bin/` directory segment. Test deltas: - `test_should_skip_path` parametrize cases switched from absolute paths to REPO_ROOT-relative inputs (matches the contract; prevents the same bug recurring). - New `test_should_skip_path_handles_running_from_worktree` regression test pins the fix. - New `test_is_entry_point_script` (10 cases) and `test_scan_skips_entry_point_runners` (end-to-end) cover the entry-point filter. 63/63 audit-script tests pass. Audit count dropped from 165 → 110 in local re-run; remaining 110 are real-or-different-FP-class candidates (lazy imports, dataclass field refs, importlib string lookups — separate follow-ups out of scope). Refs #7128 #7109 #6872 Co-authored-by: t <t@t>

mrveiss force-pushed the issue-7128 branch from 1814645 to de20b4e Compare May 7, 2026 13:55

mrveiss merged commit bbbad92 into Dev_new_gui May 7, 2026
3 of 5 checks passed

mrveiss deleted the issue-7128 branch May 7, 2026 13:56

This was referenced May 7, 2026

fix(audit-script): audit-unwired-trackers.py reports test_*.py files as unwired (#7109 sibling, #6872 follow-up) #7128

Closed

discovery(audit-script): audit-unwired-trackers.py misses feature_routers.py registry callers (#6872 follow-up) #7109

Closed

This was referenced May 7, 2026

refactor(infra/shared): re-export mocks from canonical autobot-backend SSOT (#7125) #7191

Merged

fix(audit-script): correct worktree-from-internal skip + entry-point runner filter (#7128b, #7109, #6872 follow-up) #7244

Merged

This was referenced May 8, 2026

audit: run pipeline-scripts/audit-unwired-trackers.py once across full codebase to file backlog of wire-in issues (Tier 5 of #6836) #6872

Closed

fix(audit-script): catch dynamic import() / require() / .d.ts FP classes (#6872 part 2) #7330

Merged

Navigation Menu

Repository navigation

fix(audit-script): filter test_*.py + detect router_registry callers (#7128, #7109) - #7168

fix(audit-script): filter test_*.py + detect router_registry callers (#7128, #7109)#7168
mrveiss merged 1 commit into
Dev_new_guifrom
issue-7128

mrveiss commented May 7, 2026

Uh oh!

Uh oh!

github-actions Bot commented May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mrveiss commented May 7, 2026

Summary

Changes

Result

Test plan

Uh oh!

Uh oh!

github-actions Bot commented May 7, 2026

✅ SSOT Configuration Compliance: Passing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant