Repository navigation
test(ci): 28 tests under autobot-infrastructure/shared/tests and libs/ run in no workflow, and ci.yml's integration step points at a path that does not exist #15051
Description
Activity
Partially overtaken by PR #15048 — what is already fixed
That PR covered the 12 marker-carrying tests in these trees by adding a third invocation to
marker-tests.yml, and running them for the first time surfaced defects that are fixed there
rather than left for this issue:was now test_architecture_compliance.pyerrored at collection —from utils.redis_client import get_redis_client, a module in no treerepointed at autobot_shared.redis_client, the canonical accessor CLAUDE.md mandates; the call site needed no change because the signature already matchedtest_redis_timeout_configuration—from utils.redis_helper import TIMEOUT_CONFIG, also in no treerepointed at autobot_shared.redis_management.config.PoolConfig, which carries the same four settings as typed fieldstest_concurrent_initialization_safe— asserted 6 tables where SQLite reports 7 (sqlite_sequencefor AUTOINCREMENT)asserts the named table set from one EXPECTED_TABLESconstanttest_concurrent_initialization_safe— assertedPRAGMA foreign_keys == 1on a connection the test itself opened, which is per-connection and defaults OFFasserts the schema's foreign_key_listREFERENCES clauses, which is what persists on disklibs/.../test_integration.py— 4 ×ModuleNotFoundError: autobot_sdklibs/autobot-sdk-pythonadded topytest.inipythonpathWhat this issue still owns
The 28 unmarked tests in the same trees. They are still named by no pytest invocation and no
testpathsentry, so their pass/fail split is still unknown:module unmarked test functions autobot-infrastructure/shared/tests/integration/test_architecture_compliance.py13 autobot-infrastructure/shared/tests/integration/test_distributed_system_integration.py6 autobot-infrastructure/shared/tests/performance/test_performance_optimization.py5 autobot-infrastructure/shared/tests/test_redis_db_ssot.py4 ci.yml'sRun integration testsstep still points atinfrastructure/shared/tests/integration,
which does not exist, and still passes with a::warning::on every shard-1 run.The acceptance criteria above stand unchanged, minus the two import fixes now landed.
- added 4 commits that reference this issue
on Aug 30, 2026 Closing — all six criteria met, verified in the merged tree
Landed in
c580893246(PR #15315).AC1 — the pass/fail split, re-measured
The issue's table of 28 predated PR #15048, so it was re-measured against current base before anything was changed:
module test functions marked unmarked test_architecture_compliance.py15 2 13 test_db_initialization.py6 6 0 test_distributed_system_integration.py6 6 0 test_performance_optimization.py5 5 0 test_redis_db_ssot.py4 0 4 libs/autobot-sdk-python/tests/test_integration.py5 5 0 total 41 24 17 Of the original 28, eleven gained a
pytestmarkin an unrelated change (#14979/#15166, merged after this issue was filed) and now run viamarker-tests.yml. 17 remained, and those had run in no workflow and no barepytest, ever. They ran for the first time in this PR's CI, in all 12python-suiteshards.AC2 — every failure fixed or marked with a reason
Wiring the tree in surfaced five real failures. Two were implementation bugs, fixed on the implementation side rather than by adjusting assertions:
get_backend_config()never set"host"/"port"— only the bind address"server_host"/"server_port".utils/service_discovery.py:236already read.get("host")and logged an error on every process start when it came backNone. An already-manifesting production defect, not a test artifact.test_no_hardcoded_ips_in_redis_helperimportedutils.redis_helper.REDIS_HOST— a module deleted years ago that never exported that name. Repointed at the canonicalautobot_shared.redis_client.
One test was rewritten because it was provably wrong rather than the code:
test_service_discovery_has_defaultsasserted four keys in aservice_discovery_defaultssection that two call sites already document inline as having no SSOT equivalent post-#13286.Four were marked with reasons rather than edited green: the fixed-VM-topology assertions (
test_browser_service_on_vm5,test_no_localhost_in_distributed_services,test_standard_port_assignments,test_only_one_frontend_instance). They assert a topology the platform does not have — AutoBot runs in Docker, on one VM, or on any number chosen — so they are false by construction, and rewriting them needs #15194's topology decision. Editing them to pass would have hidden the very defect wiring this file in exposed. Detail on #15194.Worth recording:
test_only_one_frontend_instancefails at "Backend must not run on frontend VM" because of theget_backend_config()fix above. Repairing a genuine bug is what made the false invariant visible.AC3–AC6
AC Verdict Evidence in origin/Dev_new_gui3. pytest.initestpathsnames both treesMET testpathslistsautobot-infrastructure/shared/testsandlibs4. ci.yml'sRun integration testsstep names a real path or is retired with a reasonMET Step is gone, with the reasoning recorded inline. It named infrastructure/shared/tests/integration— a path #734 renamed — and quietly succeeded:if [ -d "$TEST_DIR" ]; … else echo "::warning::…"had no failing branch, so it exited 0 on every shard-1 run since #734. Retired rather than repointed, because repointing would have duplicated collection and putintegration-marked tests into the required PR gate with no-mfilter5. A guard fails when a tracked test module is named by no pytest invocation, not only a marked one MET repo_tests/infra_libs_test_wiring_guard_15051_test.py6. hook_suites_run_in_ci_test.pystill passes — the four root lists agreeMET All 12 python-suiteshards green at merge; the two roots were added to all fourrepo_tests-carrying invocations to keep that agreementOne defect this PR's own CI caught, in the guard rather than the wiring
pytest_root_collection_floor_test.pywent red against a correct workflow. Its cross-check took the first barepytesttoken in arun:block, and this PR's new explanatory comment contains the wordpytestand names a step called "… infrastructure, libs and frontend" — so the parse began inside the comment and readlibsas the first root, out of prose. Fixed by stripping shell comments before tokenising and anchoring on the fullpython -m pytest, with two regression tests, one of which closes a latent second case (pip install pytest …matched the old search too).That is the same class of defect this issue is about: a check that appears to be testing something and is not.
Cross-link for #15178's AC5, recorded so the relationship survives this issue already being closed.
#15178 (the uncollected-test-file umbrella) landed in PR #15487, merged as
cf2d29d54e. Two things there bear directly on this issue:autobot-infrastructure/shared/testswas mis-classified as uncollected. It is named bypytest.ini:147,.github/workflows/ci.yml:296and.github/workflows/marker-tests.yml:341, so it is genuinely collected. It has been moved fromINTENTIONALLY_UNCOLLECTEDintoNARROWLY_COLLECTEDinrepo_tests/collection_coverage_test.py, which is why theautobot-infrastructureceiling is 51 rather than 56.- The recorded reason for excusing the wider
autobot-infrastructuretree was stale. It said the conftest importsunified_config_manager, "which no longer resolves" — that was fixed here; the import resolves viaautobot-backend/config/__init__.py:242. The exemption now records the real remaining blocker instead of repeating a spent one.
The uncollected population is now held by a down-only ratchet (
_UNCOLLECTED_CEILINGSatcollection_coverage_test.py:201) with a population floor at:212(_MIN_TRACKED_TEST_FILES = 1900) evaluated first, so a sweep that collapses to zero matches fails by name rather than reading as a clean tree. Growth fails; shrink passes.No action needed here — this is a record, not a reopen.
- added a commit that references this issue
on Sep 4, 2026
What
28 test functions under
autobot-infrastructure/shared/tests/andlibs/are named by nopytest invocation in any workflow, and by no entry in
pytest.ini'stestpaths. They runnowhere — not in the PR gate, not nightly, and not under a bare local
pytest.Measurement
Taken while implementing #13286 in PR #15048:
autobot-infrastructure/shared/tests/integration/test_architecture_compliance.pyautobot-infrastructure/shared/tests/distributed/test_db_initialization.pyautobot-infrastructure/shared/tests/integration/test_distributed_system_integration.pyautobot-infrastructure/shared/tests/performance/test_performance_optimization.pyautobot-infrastructure/shared/tests/test_redis_db_ssot.pylibs/autobot-sdk-python/tests/test_integration.pyPR #15048 covered the 12 marker-carrying ones by adding a third invocation to
marker-tests.yml, which is what #13286 is about. The other 28 are unmarked, so themarker workflow's selection deselects them and the PR gate never names their trees. Nobody
knows whether they pass.
The dead step that was supposed to run them
ci.ymlstill carries aRun integration testsstep guarded byif [ -d "$TEST_DIR" ], whereTEST_DIR="infrastructure/shared/tests/integration". That pathdoes not exist — #734 moved the suite to
autobot-infrastructure/...and the step was neverupdated. It emits a
::warning::and passes, on every shard-1 run, and has done since #734.This is the defect class #13543 groups: a step that looks like it runs something, always goes
green, and inspects nothing.
Correcting the path is not a one-line change, which is why it is filed rather than patched:
these trees' tests have never run, so an unknown number are red, and pointing the PR gate at
them would red every pull request until they are triaged.
Scope
is the argument for everything below.
integration,slow) somarker-tests.ymlpicks it up via the roots PR ci(gates): make four gates that could not fail actually fail (#13543, #13286, #13200) #15048 already added — never bydeleting or excluding a test to make a check pass.
pytest.initestpathsso a bare localpytestsees them.repo_tests-carrying invocations together, sorepo_tests/hook_suites_run_in_ci_test.py's root-set agreement stays true, and pointci.yml'sRun integration testsstep at the path that exists — or retire that step witha stated reason once the directory-based route is redundant.
Acceptance criteria
pytest.initestpathsnames both treesci.yml'sRun integration testsstep names a path that exists, or is retired with a reason recorded in the workflowrepo_tests/hook_suites_run_in_ci_test.pystill passes — the four root lists agreeProvenance
Measured while implementing #13286 in PR #15048, which covered the marker-carrying subset.
Refs #13286, Refs #13543.