Repository navigation
chore(deps): nltk is declared in two manifests but imported nowhere — remove it rather than carry a permanent allowance #15551
Description
Problem
nltk is declared twice:
requirements-ci/document.txt:6—nltk==3.10.3(pin)autobot-infrastructure/shared/docker/ai-stack/requirements-ai.txt:37—nltk>=3.10.3(floor)
Nothing imports it. Measured across the whole tree:
$ grep -rn 'import nltk\|from nltk' --include=*.py .
(0 files)
It also carries PYSEC-2026-3740 (CVE-2026-81726, GHSA-8mgp-746c-j5xp), an
advisory with first_patched_version: null — no fixed release exists. #15520
resolved the red by recording an allowance with a re-check date, because that
was the only option available to a security fix that must not change what is
installed.
A permanent allowance for a package nothing imports is the wrong end state.
Deleting the dependency removes the finding outright, and removes a download
from every CI install of requirements-ci/document.txt.
Why this is not a one-line deletion
nltk may be a transitive requirement of another declared package — the
document-processing stack around it (markdownify, beautifulsoup4,
pypdf) or the AI stack in requirements-ai.txt. If something else pulls it
in, removing the explicit declaration changes nothing about the resolved set:
the advisory stays, the allowance is still needed, and the only effect is that
the version is no longer pinned — strictly worse.
So this needs a resolution run, not an edit.
Acceptance Criteria
- Determine whether any declared dependency in
requirements-ci/document.txt
orrequirements-ai.txtrequiresnltktransitively — from a resolver's
output, not by reading requirement files - If nothing requires it: remove both declarations, and confirm CI's document
and AI installs still resolve and the document-processing tests still pass - If something does require it: record which package, on this issue, and
close it — the allowance is then correct and this is not actionable - If it is removed, delete the
--allow-id PYSEC-2026-3740line and its
comment block from.github/workflows/security.ymlin the same change.
The gate fails on an allowance matching no finding
(pipeline-scripts/security_scan_gate.py:260-268), so leaving it behind
turns this cleanup into a red - Check whether the same "declared but never imported" shape applies to the
neighbouring entries inrequirements-ci/document.txt, and file what it finds
Evidence
Found while clearing the base-level advisories for #15520 / #15540 (PR #15550).
The allowance and its reasoning are recorded in .github/workflows/security.yml
with a RE-CHECK BY 2026-12-01 date; this issue is the better resolution of the
same finding.
Activity
mrveiss commentedon Sep 4, 2026
Resolution: nltk stays. AC3 is the case that applies.
AC1 — is nltk required transitively? Yes, and by a package both manifests
declare.
llama-index-core0.14.24 listsnltk>=3.9.3in its publisheddistribution metadata (
requires_dist), which is what a resolver reads:requirements-ci/llama-index.txt:2declaresllama-index-core>=0.14.24,<0.15.0,and
requirements-ci.txtpulls in that file alongsiderequirements-ci/document.txt;autobot-infrastructure/shared/docker/ai-stack/requirements-ai.txt:9declaresthe same floor on its own, one line group above its nltk entry.
So the issue's own prediction holds exactly: removing the two declarations would
not change the resolved set. nltk installs either way; it would simply stop being
pinned, the advisory would stay reported, and the allowance would still be
required — strictly worse.
AC2 — not applicable. Nothing was removed.
AC4 — not applicable. The
--allow-id PYSEC-2026-3740line stays. Removingit would fail the gate at
pipeline-scripts/security_scan_gate.py:260-268themoment pip-audit reported the finding again.
AC5 — checked, and one finding filed: #15600. Of the thirteen other entries
in
requirements-ci/document.txt, twelve have real import sites orimportorskipgates.markdownhas none, is declared in that one file only,and no neighbour requires it — the one entry that may genuinely be removable.
What changed in PR #15599 for this issue. No declaration removed and no new
allowance. The reason is recorded at all three sites —
requirements-ci/document.txt,requirements-ai.txt, and the allowance block in.github/workflows/security.yml,whose rationale said only "nothing imports nltk". That was true but incomplete,
and read as though deletion were available; it now states why it is not. The
RE-CHECK BY 2026-12-01date is unchanged.A third site the issue did not list. The #15568 audit in the same PR found
nltk declared a third time, in ansible:
roles/dependency_patching/defaults/main.yml:43sets a>=3.9.3security flooragainst
document.txt's==3.10.3pin. That divergence is one of twelve in thesame map and is tracked in #15597, not here.
mrveiss commentedon Sep 4, 2026
Host evidence confirming the KEEP verdict, stronger than the PyPI metadata check in PR #15599.
On the deployed backend node, read from the live venv (read-only, nothing installed or changed):
So nltk is installed on the running system by
llama-index-core, which both nltk-declaring manifests also declare. Deleting the two declarations would not remove the package — it would leave it installed and unpinned, which is strictly worse than the status quo: the advisory would still be reported and the version would float. AC3 is satisfied by keeping it with the reason recorded, not by removal.Two things the host reading adds that the manifest reading could not:
==3.10.3pinned inrequirements-ci/document.txt:5. The backend venv resolves nltk transitively and no manifest on its install path pins it, so the pin is CI-only and does not describe production.first_patched_version: nulland coversnltk<=3.10.3— every release, so 3.10.0 and 3.10.3 are equally affected and there is nothing to bump to. The allowance insecurity.ymlstands as written.The version divergence itself is the class filed as #15597 (ansible floors below the shipped requirements —
dependency_patchingcarriesnltk: "3.9.3"), so it is recorded rather than dropped.fix(deps): derive the tts-worker constraints path from the base_dir S…
fix(test): name the parity baseline as a recorder so the declaration …
test(deps): exclude the session worktree copies from the parity walk (#…
chore(deps): resolve the nltk declaration and enforce ansible/require…
mrveiss commentedon Sep 4, 2026
Closed on code evidence, verified against merged
Dev_new_gui(PR #15599,c8491001).Verdict: KEEP, with the reason recorded at all three sites — removal would have been wrong.
grep -rni nltkreturns 24 hits and zero are an import. But unimported is not the same as unneededllama-index-corerequires it transitively, and both nltk-declaring manifests also declarellama-index-coresecurity.ymlrationale now records the transitive install;RE-CHECK BY 2026-12-01unchanged, no new allowance addedWhy removal was the wrong call
llama-index-core==0.14.24declaresRequires-Dist: nltk>=3.9.3. Confirmed twice, independently: against PyPI metadata, and against the deployed backend venv, wherenltk-3.10.0.dist-infois present having been pulled in transitively.So deleting the two declarations would not remove the package. It would leave it installed and unpinned — the advisory still reported, the version now floating. Strictly worse than the status quo.
The old allowance rationale ("nothing in this repository imports nltk") was true and remains true; it simply read as though deletion were available, which is what this issue was testing.
A third declaration site neither this issue nor #15568 listed:
roles/dependency_patching/defaults/main.ymlcarriesnltk: "3.9.3". All three now carry the reason.Two things the host reading added that the manifests could not
==3.10.3pinned inrequirements-ci/document.txt. No manifest on the backend's install path pins nltk, so that pin is CI-only and does not describe production.first_patched_version: nulland coversnltk<=3.10.3— every release — so 3.10.0 and 3.10.3 are equally affected and there is nothing to bump to.The version divergence itself is recorded as #15597 rather than dropped.