Repository navigation
build(python): move the Linux NPU worker to 3.14 (#13747) - #13800
Conversation
#10877 pinned this worker to Python 3.11 because OpenVINO shipped no 3.14 wheels, and that pin is what keeps black's target-version at py312 (#13748) — PEP 758 syntax would be a SyntaxError on shared code the worker imports. The premise has expired. Measured against the PyPI JSON API on 2026-08-08: openvino 2026.3.0, torch 2.13.0 and numpy 2.5.1 all publish cp314. Verified further by resolving the worker's actual requirements on 3.14 — pip solves the full graph with no conflicts, landing openvino-2026.3.0, torch-2.13.0 and numpy-2.5.1. Changed the interpreter in roles/npu-worker (the canonical path) and in deploy-native-services.yml, plus the phase in deploy-full.yml and the two provisioning playbooks whose comments named the old split. What makes this safe on an already-deployed box is ensure_venv_version, which both paths already call: it detects the existing 3.11 venv as wrong-version and removes it, so 3.11-built OpenVINO binaries are never carried into a 3.14 venv. That failure would be a native ImportError at worker startup, well after the deploy reported success. The Windows NPU worker is deliberately untouched: it pins onnxruntime-openvino, which still has no cp314 (1.24.1 ships cp311-cp313). Verified: ansible-playbook --syntax-check passes on all four changed playbooks and on provision-fleet-roles.yml, which applies roles/npu-worker, so the role graph parses. No python3.11 pin remains for the Linux worker anywhere. Does NOT close #13747: its acceptance criteria require the worker to start and serve inference on 3.14 verified on a host, and OpenVINO failures are native- extension failures that appear at import or first inference, not at pip install. That evidence has to come from a deploy.
Review — pre-merge passReviewed against What this actually changesFive files, all Ansible. Two of them build a venv and are the real change; three carry comments that named the old 3.11/3.14 split.
The risk I looked hardest atNot the fresh install — the upgrade in place. A box with an existing 3.11 venv, recreated as 3.14 over the same Both changed paths already call Checks
Judgement callI kept the separate Phase 1.5 NPU play instead of folding it into the What this PR does not doIt does not close #13747. The AC requires the worker to start and serve inference on 3.14 verified on a host; a clean dependency resolution is not that evidence, because OpenVINO fails at import or first inference rather than at install. No NPU host is reachable from here. It does not unblock #13748. That needs the worker running 3.14, not a merged config change — and the Windows-worker question is still open. VerdictNo blocking findings. |
…#13747) roles/npu-worker moved to python3.14 in #13800, but five other places still declared the pre-#13747 pin: - code_sync.py's _COMPONENT_PYTHON_TARGET (self-described as 'the single authoritative mapping'), and the test asserting its value. Currently unreachable for the npu-worker key (_WORKER_COMPONENTS routes worker components around the only two readers of this dict), but wrong data under a claim of authority is worse than no claim. - inventory/production.yml's python_version var for the npu group, with a comment restating the expired 'no 3.14 wheels' premise. - check_venv_producers.py's docstring, describing the #13746 hazard as still-live with two different interpreters. - pyproject.toml's black target-version comment and scripts/format.sh, both citing 'the 3.11 NPU worker' as the reason py314 stays unset. Corrected to the real, current blocker: #13747 needs host evidence the worker is running 3.14, not merely configured for it. None of these change target-version itself — #13748 stays gated on #13747's host verification, per the epic's explicit ordering. _COMPONENT_PYTHON_TARGET and its test file are on the file-size ratchet ceiling (6065 / 2098 lines); edits kept net-zero to stay compliant, verified with scripts/check_python_file_size.py.
Thinking Path
#10877 pinned the NPU worker to Python 3.11 because OpenVINO shipped no 3.14 wheels. That single pin is load-bearing beyond the worker: it is also why black's
target-versionstays atpy312(#13748), since PEP 758except A, B:would be a hardSyntaxErroron shared code the 3.11 worker imports.I re-measured the premise rather than trusting the recorded reason. Against the PyPI JSON API on 2026-08-08:
openvinotorchnumpyonnxruntime-openvino"openvino has a cp314 wheel" is weaker evidence than it looks, so I resolved the worker's actual requirements on 3.14 — the full graph, not one package:
No conflicts, no fallback to an sdist. That is the whole set the worker installs.
What Changed
Interpreter moved 3.11 → 3.14 in the two places that build the venv:
roles/npu-worker/tasks/main.yml— the canonical path (interpreter install,ensure_venv_version,-m venv)playbooks/deploy-native-services.yml— the documented operator path (aptpackages,ensure_venv_version,-m venv,virtualenv_python)Plus the places that described the old split: the Phase 1.5 NPU play in
deploy-full.yml, and the header comments inconfigure-python-provision-permissions.ymlandprovision-local-python.yml.The upgrade-in-place hazard, and why it is already handled
The real risk is not a fresh install — it is a box that already has a 3.11 venv. Recreating it as 3.14 over existing
site-packagesleaves 3.11-built OpenVINO native extensions in place; that fails at import or first inference, long after the deploy reports success.Both paths already call
ensure_venv_versionbefore creating the venv, which detects a wrong-version venv and removes it. That task is what makes this change safe in place, so I documented it as load-bearing rather than leaving it looking incidental.Deliberately not in scope
The Windows NPU worker (
autobot-npu-worker/resources/windows-npu-worker/) pinsonnxruntime-openvino, which has no cp314. It is untouched — confirmed no Windows file appears in this diff.I also kept the separate Phase 1.5 NPU play rather than folding it into the
backend:aimlone.npuis a distinct host group that pattern does not reach, and thepython314role is idempotent, so a co-located host just skips the second install.Verification
No
python3.11pin remains for the Linux worker anywhere inansible/. The one surviving mention is a parameter example inensure_venv_version.yml's docstring, which documents the argument's form and is not a pin.This PR does not close #13747
Its acceptance criteria require the worker to start and serve inference on 3.14, verified on a host — explicitly not by a successful
pip install. OpenVINO failures are native-extension failures that surface at import or first inference, so a green resolution proves the dependency set solves, not that the NPU runs.I have no NPU host to verify against, so the issue stays open pending that deploy evidence. Merging this is what makes the next NPU deploy use 3.14; confirming it worked is a separate, host-side step.
#13748 (black → py314) remains blocked and must not be taken as unblocked by this merge. It is unblocked only once the worker is running 3.14 — reversing that order writes PEP 758 syntax into shared code a still-3.11 worker imports, which is a
SyntaxErrorat import, in production.Model Used
Claude Opus 5 (1M context)
Part of #13743