Skip to content

ci: autobot-infrastructure/shared/tests cannot be collected on its own — a namespace-package config shadows the backend's #15161

Closed
Closed
ci: autobot-infrastructure/shared/tests cannot be collected on its own — a namespace-package config shadows the backend's#15161
Labels
bugSomething isn't workinginfrastructurepriority: high

Description

@mrveiss
mrveiss
opened on Aug 28, 2026

Problem

autobot-infrastructure/shared/tests/conftest.py fails to import, so every
collection under that tree errors before a single test is found:

ImportError while loading conftest '.../autobot-infrastructure/shared/tests/conftest.py'.
E   ImportError: cannot import name 'unified_config_manager' from 'config' (unknown location)

Two independent causes, both in the same six lines (conftest.py:17-21):

project_root = Path(__file__).parent.parent.parent.parent
sys.path.insert(0, str(project_root / "autobot-user-backend"))

from config import unified_config_manager
  • autobot-user-backend/ does not exist in the repository, so the
    sys.path.insert adds a non-existent directory.
  • unified_config_manager is not exported by any config module anywhere in the
    tree (find . -name "unified_config_manager*" returns nothing).

Why it matters

.github/workflows/marker-tests.yml:294 names
autobot-infrastructure/shared/tests as a pytest root. That invocation is
therefore returning a conftest ImportError rather than running tests, and has
been for as long as the conftest has been broken — a whole declared CI root is
silently contributing zero.

This is the #14917 family exactly: a tree that looks wired into CI, is named in a
workflow, and reaches nothing.

How it was found

While converting files for #14979. Two of that issue's fourteen files live under
this tree:

  • autobot-infrastructure/shared/tests/performance/test_performance_optimization.py
  • autobot-infrastructure/shared/tests/integration/test_distributed_system_integration.py

Both were converted correctly and carry valid markers (performance,
distributed) that marker-tests.yml selects for. Neither is actually
reachable in CI while this conftest is broken, so their conversion is
knowingly cosmetic until it is fixed. They were verified locally with
--noconftest -c pytest.ini --rootdir=..

Secondary finding

autobot-infrastructure/shared/tests/pytest.ini registers integration, slow
and distributed but not performance. CI roots at the repository root so
this does not bite there, but any invocation rooted in that directory fails under
--strict-markers.

Acceptance criteria

  • pytest autobot-infrastructure/shared/tests --collect-only completes without a conftest error
  • The autobot-user-backend path insert is removed or repointed at a directory that exists
  • unified_config_manager is imported from wherever the canonical config manager actually lives, or the fixture that needed it is removed
  • performance is registered in that tree's pytest.ini
  • A guard asserts each pytest root named in marker-tests.yml collects a non-zero number of items — a root that collects nothing must fail, not pass quietly
  • The two bug(test): 90 test_* methods sit in live-service validation drivers pytest cannot collect #14979 files above are confirmed collected and selected by the marker run

Related

Activity

  1. mrveiss
    added
    bugSomething isn't working
    priority-high
    on Aug 28, 2026
  2. github-actions
    added
    infrastructure
    on Aug 28, 2026
  3. mrveiss
    added 2 commits that reference this issue on Aug 28, 2026

    style(ci): black-clean the collection-floor guard test (#15161)

    3d60d52

    docs(ci): state the collection-floor contract exactly (#15161)

    fd713a6
  4. mrveiss
    changed the title [-]ci: autobot-infrastructure/shared/tests collects nothing — conftest imports a module and a directory that do not exist[/-] [+]ci: autobot-infrastructure/shared/tests cannot be collected on its own — a namespace-package `config` shadows the backend's[/+] on Aug 28, 2026

  5. mrveiss commented on Aug 28, 2026

    @mrveiss
    mrveiss
    on Aug 28, 2026
    OwnerAuthor

    Correcting this issue — I filed it wrong

    The headline claim was false, and I am retitling rather than quietly editing.

    I wrote that "every collection under that tree dies with ImportError" and that
    marker-tests.yml:294 was therefore "returning a conftest ImportError rather than
    running tests". That is not true. Measured with the CI-shaped invocation, which
    names both roots the workflow names:

    $ pytest autobot-infrastructure/shared/tests libs \
    -m "integration or slow or distributed or performance" --collect-only
24/41 tests collected (17 deselected)

    No ImportError. The workflow's own comment agrees and I should have believed it over
    my local run: that invocation is recorded as 12 collected / 10 passed / 0 failed.

    What I actually hit is narrower and still real: the failure only occurs when that
    directory is named alone.

    $ pytest autobot-infrastructure/shared/tests --collect-only
ImportError: cannot import name 'unified_config_manager' from 'config' (unknown location)

    (unknown location) is the tell — it is a namespace package, not the real module.
    autobot-infrastructure/shared/config/ is a directory of YAML with no __init__.py,
    so when that tree is the sole root it can be imported as a namespace package named
    config and shadow autobot-backend/config, which is what actually exports
    unified_config_manager (see autobot-backend/conversation_file_manager.py:38).
    Adding a second root changes resolution and the correct module wins.

    So the conftest is not broken, autobot-user-backend being absent is inert (the
    sys.path.insert of a non-existent directory is harmless), and no CI root is dead.

    What remains genuinely wrong

    1. A developer running that tree directly gets a misleading ImportError naming a
      symbol rather than the shadowing that caused it. It is order-dependent and
      environment-dependent, which is the worst shape for a newcomer.
    2. The dead sys.path.insert for autobot-user-backend, a directory in no
      commit, should go regardless — it exempts nothing and implies a tree that
      does not exist.
    3. performance was missing from that tree's pytest.ini markers (it registered
      integration, slow, distributed). Under --strict-markers any invocation
      rooted there turned a performance-marked module into a collection error. Fixed
      in test(suite): convert the thirteen uncollected validation drivers and settle all three ceilings (#14979) #15166      , which needed it for
      shared/tests/performance/test_performance_optimization.py.

    Corrected acceptance criteria

    • pytest autobot-infrastructure/shared/tests alone collects without an ImportError
      — most likely by giving autobot-infrastructure/shared/config/ an __init__.py,
      renaming it, or excluding it from import, whichever is right for that directory
    • The autobot-user-backend sys.path.insert is removed from
      shared/tests/conftest.py
    • performance registered in that tree's pytest.ini (landed in test(suite): convert the thirteen uncollected validation drivers and settle all three ceilings (#14979) #15166)
    • A guard asserts each pytest root named in marker-tests.yml collects a non-zero
      number of items — still worth having, and it is the part of my original filing
      that survives

    Correction to #15166

    The two converted files under this tree — test_performance_optimization.py (5) and
    test_distributed_system_integration.py (6) — are not blocked by this issue. Both
    collect and are marker-selected under the real root list, verified above. I had
    described them as unreachable; that was wrong and the PR body is being corrected.

    Apologies for the noise — the original filing generalised one bad local invocation
    into a claim about CI without checking CI.

  6. mrveiss
    added a commit that references this issue on Aug 28, 2026

    fix(ci): make autobot-infrastructure/shared/tests collect, and floor …

    df2ce22

  7. mrveiss commented on Aug 29, 2026

    @mrveiss
    mrveiss
    on Aug 29, 2026
    OwnerAuthor

    Verified in merged Dev_new_gui — closing

    Merged as df2ce22a20 (PR #15175). Verified against the corrected criteria from the 2026-08-28 comment, which supersede the body.

    Criterion Verdict Evidence
    The tree collects on its own without an ImportError MET autobot-infrastructure/shared/tests/conftest.py now appends autobot-backend — a real package — instead of the nonexistent autobot-user-backend, so from config import unified_config_manager resolves whatever the rootdir. Measured: exit 4 / 0 items → exit 0 / 28 items, with the "before" reproduced from the committed base file
    The dead autobot-user-backend path insert is removed MET Only an explanatory comment remains at :16; no path insert survives anywhere in that tree
    The performance marker is registered MET shared/tests/pytest.ini:28
    A guard fails if any root named in marker-tests.yml collects zero MET pipeline-scripts/pytest_root_collection_floor.py, wired at .github/workflows/marker-tests.yml:384, with 22 tests. Roots are derived from the workflow, not hand-listed, so a root added there cannot silently escape the floor

    That last detail is what makes the guard durable. A hand-maintained list of roots would drift the moment someone added a twelfth; deriving them from the workflow means the guard's population follows the thing it guards. It ran green across all 12 roots (27,910 items) and its contrast mutation flipped it red.

    Minor, not blocking: pytest.ini now registers performance twice (:28 and :30). Harmless duplicate, worth a one-line tidy whenever that file is next touched.

  8. mrveiss
    closed this as completedon Aug 29, 2026

  9. mrveiss commented on Sep 2, 2026

    @mrveiss
    mrveiss
    on Sep 2, 2026
    OwnerAuthor

    Cross-link for #15178's AC5, recorded so the relationship survives this issue already being closed.

    #15178 (the uncollected-test-file umbrella) landed in PR #15487, merged as cf2d29d54e. Two things there bear directly on this issue:

    1. autobot-infrastructure/shared/tests was mis-classified as uncollected. It is named by pytest.ini:147, .github/workflows/ci.yml:296 and .github/workflows/marker-tests.yml:341, so it is genuinely collected. It has been moved from INTENTIONALLY_UNCOLLECTED into NARROWLY_COLLECTED in repo_tests/collection_coverage_test.py, which is why the autobot-infrastructure ceiling is 51 rather than 56.
    2. The recorded reason for excusing the wider autobot-infrastructure tree was stale. It said the conftest imports unified_config_manager, "which no longer resolves" — that was fixed here; the import resolves via autobot-backend/config/__init__.py:242. The exemption now records the real remaining blocker instead of repeating a spent one.

    The uncollected population is now held by a down-only ratchet (_UNCOLLECTED_CEILINGS at collection_coverage_test.py:201) with a population floor at :212 (_MIN_TRACKED_TEST_FILES = 1900) evaluated first, so a sweep that collapses to zero matches fails by name rather than reading as a clean tree. Growth fails; shrink passes.

    No action needed here — this is a record, not a reopen.

  10. mrveiss
    added
    priority: high
    and removed
    priority-high
    on Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinginfrastructurepriority: high

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions