Consolidate Python dependency versions into a single constraints file (unblock numpy 2.x + py3.13 in backend) #10524

Goal

Establish a single source of truth for shared/transitive Python dependency versions so every component installs the same, latest version of common libraries (numpy first), and bumping a version is a one-line change. Today numpy is pinned in 6 files with 3 different floors, and the backend is silently held a major version behind.

Why this matters now — portable roles

AutoBot roles (backend, tts-worker, npu-worker, ai-stack, code_analysis) are relocatable across hosts and can be co-located on one host by Ansible. When two roles with conflicting pins land in the same Python environment, pip cannot satisfy both. Version skew also breaks the array/serialization ABI when a .npy/pickled embedding written by a numpy-2.x role is read by a numpy-1.x role. Single version is a co-location correctness requirement, not just hygiene.

Current state (evidence)

numpy declared in 6 files, 3 different floors:

The main backend does NOT pin numpy directly; it is forced to numpy<2 transitively, via a constraint hidden on the opencv line (autobot-backend/requirements.txt:42). No constraints file exists anywhere in the repo.

Root cause / blocker

graspologic>=3.4.4 (autobot-backend/requirements.txt:51) is a double blocker — latest stable 3.4.4 declares:

numpy<2.0.0,>=1.26.4
requires_python: <3.13,>=3.9

So it caps the backend at numpy<2 and cannot install on Python 3.13+ (collides with the py3.14 migration). There is no numpy-2 graspologic release (even in dev). It is used in exactly one place — two leiden(G, trials=N) calls in autobot-backend/services/mesh_brain/community_clusterer.py:52,93, over a networkx graph, returning {node: community_id}.

Tasks

• Task 1: Replace graspologic Leiden → PR refactor(mesh-brain): replace graspologic Leiden with NetworkX Louvain — unblock numpy 2.x + py3.13 (#10524) #10530
  • Subtask 1.1: Swap the two leiden() calls in services/mesh_brain/community_clusterer.py for a numpy-2 + py3.13-compatible equivalent — leidenalg+python-igraph (true Leiden) OR networkx.community.louvain_communities (already a dep, no new package). Recommend leidenalg for algorithmic fidelity; confirm choice with board.
  • Subtask 1.2: Preserve the {node: community_id} return contract + existing fallback/exception behavior; keep cluster_graph / _split_community behavior identical.
  • Subtask 1.3: Tests for community clustering on a known graph (stability of centroids).
  • Subtask 1.4: Remove graspologic>=3.4.4 from autobot-backend/requirements.txt; drop the opencv-python-headless<4.11 cap (allow latest opencv / numpy>=2).
• Task 2: Single constraints file (source of truth) → PR build(deps): single-source shared dep versions via constraints/shared.txt + drift guard (#10524) #10556
  • Subtask 2.1: Add constraints/shared.txt at repo root with one pin per shared lib (numpy==<latest>, scipy, opencv, etc.).
  • Subtask 2.2: Reference it from every component requirements file (-c <relpath>/constraints/shared.txt) and remove the per-file numpy version, leaving bare numpy.
  • Subtask 2.3: Verify each component (backend, tts-worker, npu-worker, windows-npu-worker, ai-stack, code_analysis) resolves and builds against the shared constraint.
• Task 3: Co-location guard → PR build(deps): single-source shared dep versions via constraints/shared.txt + drift guard (#10524) #10556
  • Subtask 3.1: CI check (or pre-commit) that fails if any requirements file declares a version for a lib listed in constraints/shared.txt (prevents re-drift).
  • Subtask 3.2: Document in dependency docs that portable/co-locatable roles must share the constraints file.

Verification

• All component images build on the unified numpy (CPU default stays CPU-only; GPU extra unchanged).
• community_clusterer tests pass with the new partitioner.
• Backend imports on Python 3.13+ (graspologic removed).
• Grep shows numpy version declared in exactly one place.