fix(infra): infrastructure scripts stop depending on the caller's PYTHONPATH, one guard for the class, and one monitoring launcher (#18288, #17906)

[mrveiss]

Thinking Path

#18288: infrastructure scripts imported backend internals (utils.* and others) and resolved them only when the caller's PYTHONPATH already held the backend. Run standalone, they failed. The owner asked for the fix as soon as possible. Two of the ten scripts are edited by open #18230 (same-file rule), so they follow on this branch after it merges. Two others share files with the unfinished local branch for #17906 (merging the two monitoring launchers), so that work rides here too.

A class guard written for this PR found the problem is 104 modules, not 10. The 94 not audited here are frozen in its allowlist and filed as #18289.

What Changed

• c3f84fef8d, f7cac37817, 995d41ba54 (refactor(infrastructure): consolidate the two near-duplicate monitoring launchers into one #17906): one monitoring launcher. start_monitoring.py --backend hardware replaces start_hardware_monitoring.py, which is removed: a full-repo grep found no caller. Backend selection lives in a sibling monitoring/monitoring_backends.py, which keeps the launcher at 532 lines. The monitoring floors drop to 1 in the same commit. Cherry-picked onto current main; main has changed none of its files since.
• 05aa29e638 (infra: ten infrastructure scripts import backend internals and fail standalone — one guard for the class #18288): each script is decided as one of three: shared code moved to autobot_shared, a backend tool, or dead. All seven came out as backend tools, and each now puts autobot-backend and the repo root on its own path.

Script	Clean python -I import before	Why a backend tool
performance_dashboard.py	No module named 'utils'	drives monitoring_system, which needs the backend's config and constants
validation_dashboard_generator.py	No module named 'project_state_tracker'	reports on backend project-state modules; called by phase_validation.yml
test_phase5_cleanup.py	lazy import, so it failed only when run	checks backend metrics collectors
test_long_running_operations.py	No module named 'autobot_shared'	demo of the backend long-running framework, which always fell back to "unavailable"
apply_memory_optimizations.py	No module named 'utils'	utils.memory_optimization needs psutil, which autobot_shared does not declare
analysis/test_service_registry.py	lazy import	deliberately exercises the backend registry
monitoring/start_monitoring.py + monitoring_backends.py	No module named 'autobot_shared'	drives backend monitors

All of them now return 0 on a clean import.

• Guard (repo_tests/infra_scripts_no_backend_imports_18288_test.py + infra_backend_imports_allowlist_18288.py):
  • It is a syntax-tree scan of every tracked .py under autobot-infrastructure/, function bodies included.
  • The backend-only names (223) are derived from autobot-backend's top level, minus the standard library and root-provided names. There is no hand list.
  • Backend tools must also import cleanly in a subprocess.
  • Reach floor: infra-backend-import-scan. There are 8 contrast fixtures.
  • The allowlist only shrinks: a stale entry fails. It holds the 7 tools, 2 entries pending fix(knowledge): one active embedding model resolver, model-aware cache keys, and search refuses rather than embed in another space (#18143, #18185) #18230, 1 pending refactor(infra): one service-registry core, one health status and one deployment-mode detector for the backend and the deployment scripts (#18132, #12771) #18287, and 94 unclassified entries owned by infra: 98 more infrastructure modules import backend-only packages — classify each as shared, backend tool or dead (#18288 follow-up) #18289.
• repo_tests/infra_monitoring_import_path_15643_test.py now also accepts a sibling <name>.py module as a resolvable import, not only a package directory. Review this widening.

Behaviour changes

• start_hardware_monitoring.py is gone. Anyone invoking it by hand on a host switches to start_monitoring.py --backend hardware.
• test_long_running_operations.py now really loads the framework instead of always reporting it unavailable.

Verification

• New guard plus the 17906 and 15643 tests: 34 passed. Infra tests (apply_memory_optimizations, validation_dashboard_generator): 6.
• reach_scope_claim_17844 alone 10, guard-reach meta 23, reach_declarations 411, symbol-fork 17, size ratchet 31, ceiling parity 11.
• flake8 is clean. No script was run against a live system; the checks are import-only.

Not done, stated rather than implied

• verify_knowledge_consistency.py and analysis/npu_performance_measurement.py are added to this branch after fix(knowledge): one active embedding model resolver, model-aware cache keys, and search refuses rather than embed in another space (#18143, #18185) #18230 merges.
• Whichever of refactor(infra): one service-registry core, one health status and one deployment-mode detector for the backend and the deployment scripts (#18132, #12771) #18287 and this PR merges second deletes the stale monitor_services.py allowlist entry.
• The 94 modules are infra: 98 more infrastructure modules import backend-only packages — classify each as shared, backend tool or dead (#18288 follow-up) #18289.
• Two bugs were found by reading, not running: bug(backend): template_loader resolves templates/ at the repository root, where it does not exist — dashboard CSS never loads #18290 (missing templates directory) and bug(backend): api/validation_dashboard imports scripts.validation_dashboard_generator from the repository root, where it is not — the generator is always a MissingDep #18291 (the validation dashboard imports its generator from the wrong place).

So this PR is Refs #18288, and Closes #17906.

Closes #17906
Refs #18288

Model Used

Claude Opus 5.5 (master session: brief, review of the report, PR). Implementation by a Claude Opus subagent.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: e9873621-dcb2-4fd5-ad99-5a1c3ad64e28

---

📥 Commits

Reviewing files that changed from the base of the PR and between 1349f43 and 48d1a9f.

---

📒 Files selected for processing (24)

• .github/workflows/ci.yml
• .github/workflows/coverage.yml
• .github/workflows/marker-tests.yml
• .github/workflows/test-durations.yml
• autobot-infrastructure/shared/scripts/analysis/test_service_registry.py
• autobot-infrastructure/shared/scripts/apply_memory_optimizations.py
• autobot-infrastructure/shared/scripts/monitoring/monitoring_backends.py
• autobot-infrastructure/shared/scripts/monitoring/start_hardware_monitoring.py
• autobot-infrastructure/shared/scripts/monitoring/start_monitoring.py
• autobot-infrastructure/shared/scripts/monitoring/start_monitoring_category_default_test.py
• autobot-infrastructure/shared/scripts/performance_dashboard.py
• autobot-infrastructure/shared/scripts/test_long_running_operations.py
• autobot-infrastructure/shared/scripts/test_phase5_cleanup.py
• autobot-infrastructure/shared/scripts/validation_dashboard_generator.py
• docs/architecture/LONG_RUNNING_OPERATIONS_ARCHITECTURE.md
• pytest.ini
• repo_tests/_infra_backend_names.py
• repo_tests/_reach_policy.py
• repo_tests/infra_backend_imports_allowlist_18288.py
• repo_tests/infra_monitoring_import_path_15643_test.py
• repo_tests/infra_scripts_no_backend_imports_18288_test.py
• repo_tests/monitoring_launcher_single_17906_test.py
• repo_tests/reach_scope_claim_17844_test.py
• repo_tests/test_infra_scripts_import_14129.py

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

[mrveiss]

Merger review at 995d41b: BLOCKED. Two must-fix items, then two guard tightenings.

These check out:

• the launcher with --backend hardware, whose hardware branch the tests prove is taken;
• no deploy-time caller of the removed shim (sh, yml, j2, systemd, ansible, SLM, Makefile and package.json all checked);
• the class guard's derived backend-name set, its 8 contrasts, the reach floor (266 against 281), and the stale-entry failure that enforces the refactor(infra): one service-registry core, one health status and one deployment-mode detector for the backend and the deployment scripts (#18132, #12771) #18287 interlock.

Sev	Where (at 995d41b)	Finding	Fix
HIGH	repo_tests/test_infra_scripts_import_14129.py:84	_OPERATOR_ENTRYPOINTS still lists monitoring/start_hardware_monitoring.py, which this PR deletes. Its test asserts script.is_file(), so it goes red. It is a test-side caller that the "no caller" sweep missed.	Point it at monitoring/start_monitoring.py, and update the docstring mentions at :8-10, :42 and :142.
HIGH	#17906 AC4 / pytest.ini testpaths	autobot-infrastructure/shared/scripts/monitoring/start_monitoring_category_default_test.py runs nowhere: testpaths includes only autobot-infrastructure/shared/scripts/hooks. The AC asks for it to run, or for its not running to be tracked. The PR says Closes #17906.	Wire the directory into testpaths the way hooks was (#13880), and measure it green. Otherwise file the tracking issue and change the PR to Refs #17906.
MED	infra_monitoring_import_path_15643_test.py:~160 (the change you asked about)	Accepting any sibling <name>.py as resolvable admits false negatives. The launcher inserts its own directory at index 0, so a future utils.py or constants.py beside it would shadow the backend module and still pass.	Accept a sibling only when the root is the script's own directory and the name is not in backend_only_names(). Also insert the monitoring directory after the backend root, not at index 0.
MED	infra_scripts_no_backend_imports_18288_test.py _own_tree_names	A backend name is suppressed when any ancestor directory up to autobot-infrastructure/ has a same-named entry, but Python only puts the script's own directory on the path. Twelve top-level names are shared between infra and backend (api, knowledge, monitoring, security, tools, user_management, …), so a real backend import of one of those names would be hidden.	Suppress only names that exist in the script's own directory or in a directory the script inserts into sys.path.
LOW	the guard's AST scan	importlib.import_module("x") and __import__("x") calls with a constant first argument are not scanned.	Add them with a contrast fixture, or record them on #18289.
LOW	start_monitoring.py main()	About 65 lines, over the 30-line rule; it was already long.	Extract the dispatch and config loading.

Unverified: whether hardware_metrics.py's Redis client is created lazily. The launcher now imports backend modules that build singletons at import time (gpu_acceleration_optimizer.py:363, hardware_metrics.py:1332, performance_monitor.py:135).