Skip to content

fix(test): stop the SLM version-check tests reading the host code_source (#13162) - #13371

Merged
mrveiss merged 1 commit into
Dev_new_guifrom
issue-13162-slm
Aug 3, 2026
Merged

mrveiss merged 1 commit into
Dev_new_guifrom
issue-13162-slm

Conversation

@mrveiss

@mrveiss mrveiss commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Thinking Path

Scope was autobot-slm-backend/migrations/ and autobot-slm-backend/tests/services/, reported as ~23 CI failures split 18 / 5.

Baseline on the dev box (Python 3.10, SLM-only pytest invocation) was 428 passed, 1 skipped, 0 failed — nothing reproduced. Rather than guess at a Python 3.14 difference, I pulled the actual shard logs from the latest Dev_new_gui CI run and grepped them. Only five failures exist in this scope, all in version_checker_test.py::TestVersionCheckTask, and every one of them is an assertion that check_for_updates / update_latest_version_setting / the "Update available" log never happened.

That pattern points at one shared precondition, not five bugs. version_check_task resolves the active CodeSource path, then gates its entire body on _is_git_repo(repo_path) — a deliberate skip added in #9716 for deployments with no local checkout. The tests patch get_git_tracker and db_service, but never the path resolution, so the loop fell through to the module's DEFAULT_REPO_PATH and read the host filesystem. The dev box has a live install at that location, so the gate opened and the tests passed. A CI runner does not, so the gate closed and all five assertions failed.

Confirmed the diagnosis by reproducing the exact CI failure set locally with SLM_REPO_PATH=/nonexistent/code_source: 5 failed, 4 passed — the same five test IDs. So this is not a Python 3.14 artifact; it is environment coupling that happens to be invisible on any machine with a deployed checkout at the default location.

The reported 18 failures in migrations/ are not real. That directory is green in both environments and in CI. They reproduce only when autobot-backend and autobot-slm-backend are collected in a single pytest session — both define a top-level migrations package, so whichever collects first binds the dotted name and the other's imports fail. This is the exact hazard the comment above testpaths in pytest.ini documents, and CI already runs the two backends as separate invocations. Demonstrated below.

What Changed

autobot-slm-backend/tests/services/version_checker_test.py only. No production code was modified — the _is_git_repo skip is intended behaviour and the tests were wrong to depend on the host for it.

  • Added an autouse fixture on TestVersionCheckTask that points _get_active_code_source_config at a tmp_path tree containing a real .git directory. The genuine _is_git_repo check still runs (it is not stubbed out), it just now has a deterministic tree to look at. This also removes the incidental real-DB session two of the cases were attempting. Mirrors the isolation pattern the sibling git_tracker_test.py already uses.
  • Reset the _missing_repo_warned module global per test so the once-only log flag cannot leak between cases.
  • Added test_task_skips_when_no_local_checkout, pinning the gate itself. The loop's skip path had no coverage — only _run_git_command's equivalent did — which is precisely why the gate could silently disable five tests without anyone noticing.

No test was skipped, xfailed, or weakened, and no assertion was relaxed.

Verification

SLM-only pytest invocation throughout, per the pytest.ini note (autobot-backend and autobot-slm-backend must not share a session).

Reproducing the CI environment, SLM_REPO_PATH pointed at a path with no checkout — before:

FAILED version_checker_test.py::TestVersionCheckTask::test_task_calls_check_for_updates
FAILED version_checker_test.py::TestVersionCheckTask::test_task_updates_setting_on_success
FAILED version_checker_test.py::TestVersionCheckTask::test_task_handles_no_remote_commit
FAILED version_checker_test.py::TestVersionCheckTask::test_task_handles_exception
FAILED version_checker_test.py::TestVersionCheckTask::test_task_logs_update_available
5 failed, 4 passed in 2.01s

Same command, after:

10 passed, 1 warning in 1.65s

Full scope on the dev box, after:

429 passed, 1 skipped, 2 warnings in 36.37s

(428 before, plus the one new regression test.)

The new test is mutation-verified. Replacing the if not _is_git_repo(repo_path): gate with a constant-false condition makes it fail:

FAILED version_checker_test.py::TestVersionCheckTask::test_task_skips_when_no_local_checkout
1 failed in 0.63s

The mutation was reverted; git status shows the test file as the only modification.

migrations/ needs no change — 28 tests, all passing:

28 passed in 20.56s

And the collision that produced the phantom 18 is a single-session artifact. Collecting an autobot-backend migrations test alongside the SLM ones makes the backend side lose the name race here, with the SLM 28 still green:

FAILED autobot-backend/tests/migrations/test_db_url.py::test_get_url_dev_fallback_default_host
FAILED autobot-backend/tests/migrations/test_db_url.py::test_get_url_dev_fallback_custom_host
FAILED autobot-backend/tests/migrations/test_db_url.py::test_get_url_dev_fallback_does_not_pick_up_other_pg_vars
FAILED autobot-backend/tests/migrations/test_db_url.py::test_get_url_prefers_autobot_database_url_env_var
4 failed, 28 passed in 17.67s

Reverse the collection order and the 18 SLM ones lose instead. CI does not run them together, so neither set is actually red there.

Lint on the changed file, all clean: isort (line length 120), black (line length 120, unchanged), flake8 (exit 0), bandit (no issues identified, no severity floor).

Model Used

Claude Opus 5

Closes #13162

PARTIAL. This PR fixes only the autobot-slm-backend/migrations/ and autobot-slm-backend/tests/services/ group. Other failure groups in the suite are being handled separately, so #13162 stays open until every group has landed.

…rce (#13162)

The five TestVersionCheckTask cases never established the precondition the
loop they exercise requires. version_check_task skips its whole body when the
active CodeSource path has no .git directory (#9716), and the tests patched
only get_git_tracker / db_service — never the path resolution. So the loop
fell through to the module's DEFAULT_REPO_PATH and read whatever the host
happened to have there: green on a developer box with a live install,
red on a CI runner without one. That is the entire cause of the five CI
failures; the production skip is deliberate and stays as is.

Give the class an autouse fixture that points _get_active_code_source_config
at a tmp_path tree containing a real .git directory, so the genuine
_is_git_repo check runs and passes on any host, and reset the
_missing_repo_warned module global between cases. This mirrors the pattern
the sibling git_tracker_test.py already uses.

Add test_task_skips_when_no_local_checkout to pin the gate itself. The loop's
skip path had no coverage — only _run_git_command's did — which is why the
gate could silently disable five tests. Verified by mutation: removing the
gate makes the new test fail.

Verification (SLM-only pytest invocation, per the pytest.ini note):
- before, with no host checkout: 5 failed, 4 passed
- after, with no host checkout: 10 passed
- after, full scope, migrations + tests/services: 429 passed, 1 skipped

autobot-slm-backend/migrations is already green in both environments; its 28
tests pass. The 18 reported failures there reproduce only when the two
backends are collected in one pytest session, where whichever collects first
binds the shared migrations dotted name. CI already runs the two backends as
separate invocations, so no change is needed.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!

@mrveiss
mrveiss merged commit 17db271 into Dev_new_gui Aug 3, 2026
49 of 62 checks passed
@mrveiss
mrveiss deleted the issue-13162-slm branch August 3, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant