ci: production and CI dependency sets are hand-mirrored, and drift is silent #14551

Summary

Dependencies are declared twice, in two files that must be kept in sync by hand:

• autobot-backend/requirements.txt — what production installs
• requirements-ci/*.txt — what CI installs (setup-python-suite runs pip install -r requirements-ci.txt; it never installs the backend's own requirements file)

A package declared in one and not the other produces no error. Production-only means the tests never exercise it; CI-only means it passes in CI and is missing at runtime.

Evidence

pytesseract was declared in autobot-backend/requirements.txt by #13885 and was still absent from CI when #13896 landed. Every end-to-end OCR test had been skipping the whole time, and the suite was green throughout. The gap surfaced only because #13896 added an explicit provisioning assertion:

AssertionError: OCR toolchain missing on the CI runner: pytesseract is not installed.

The split is deliberate and documented (#1655, role-based grouping to contain version conflicts), and the files themselves show the manual sync:

requirements-ci/ai-ml.txt:9:  # Mirrors autobot-backend/requirements.txt:42.
requirements-ci/utilities.txt:10: apscheduler>=3.11.3,<4.0.0   # production dep (autobot-backend/requirements.txt)

That second comment goes on to describe the same class of latent failure — "Errors nothing today only because lifespan.py imports that module inside a function — same latent shape as croniter (#13351)". So this has bitten before.

What is not being proposed

Merging the two files. The role-based split exists for a reason and CI legitimately omits heavy production deps (torch arrives separately, per #13316). The problem is not that they differ — it is that they differ silently.

Scope

• A guard that compares the two sets and reports packages present in production but absent from CI
• An explicit, documented allowlist for the deliberate omissions (torch, GPU-only, anything CI genuinely should not install) — so the diff is expected to be empty and any new entry is a decision, not an oversight
• Run it in code-quality, where the other convention checks live
• Seed the allowlist from today's actual diff, only-ever-shrinking, in the style of sys_modules_leak_baseline.txt and extension_import_baseline.txt

Why it is worth the guard

The failure mode is invisible by construction: an optional import degrades to a message at runtime and to a skipif in tests, so the missing package produces a green suite in both places. That is the same blindness behind #13885 and, differently, #14550.

Related

• fix(deps): pytesseract binding never installed — every OCR surface is dormant #13885 — declared in production, absent from CI for months
• feat(document): rasterize-then-OCR fallback for PDFs with no text layer #13896 — where it surfaced, and adds the assertion that surfaced it
• ci: system packages ansible installs on hosts are absent from CI runners, so their tests skip silently #14550 — the system-package counterpart (apt packages on hosts, absent from runners)
• fix(ci): resolve dependency conflicts and split requirements-ci.txt into role-based files #1655 — the original role-based CI split