Repository navigation
bug(slm): the whole-machine self-update path has no disk gate and no rollback, though both exist elsewhere in the repo #17262
Description
Problem
ansible/playbooks/update-all-nodes.yml (2,260 lines) — the playbook actually run by POST /self-update and by stage 3 of POST /update-all — has no free-disk gate and no rollback. Both mechanisms exist in this repo; neither is connected to this path.
Verified directly: grep -niE 'rollback|snapshot|backup|disk|free_space|dependency_patching' over that playbook returns only five unrelated prose comments ("already on disk", "the file on disk"). No task.
The machinery exists elsewhere
| Mechanism | Where it works | Reaches the self-update path? |
|---|---|---|
| Free-disk gate | ansible/apply-system-updates.yml:29-38 (fails under 5 GB), playbooks/pre-deployment-validation.yml:29-30,136-158 (10 GB min) |
no — the latter is imported only by provision_host.yml / deploy-full.yml |
| Venv backup + rollback | roles/dependency_patching via playbooks/patch-dependencies.yml / rollback-dependencies.yml, driven by ansible/deploy.sh and the admin route POST /settings/updates/run (autobot-backend/api/settings.py) |
no |
| Component snapshot + health-gated revert | api/code_sync.py:2467-2584; the health gate at :2592-2636 correctly refuses to revert a slow-but-healthy restart |
no — called only from the per-component drift-resolve path (:3032), never from _ansible_self_update |
So a whole-machine self-update that exhausts the disk mid-run, or that leaves the node broken, has no automatic path back.
Note on scope
This is a wiring gap, not missing machinery, and nothing here is orphaned. #17257 was closed for asserting dependency_patching was invoked by nothing — that claim was false (the role is invoked from patch-system-packages.yml:70,85,100,115,130,145, rollback-system-packages.yml:59,71, deploy.sh and an admin API route). This issue makes the narrower, verified claim: those mechanisms are not reachable from update-all-nodes.yml.
Acceptance criteria
-
update-all-nodes.ymlfails before mutating anything when free space is below a configured floor - The floor is derived from the actual size of what is about to be installed, not a fixed guess, wherever that size is knowable
- A failed whole-machine self-update has an automatic path back, or the issue documents explicitly why it cannot and what the operator must do instead
- The revert reuses the existing health gate so a slow-but-healthy restart is never wrongly reverted
- A test covers "insufficient disk → refused before mutation"
Activity
mrveiss commentedon Sep 25, 2026
Independent rediscovery in a comparative platform audit — a witness that raises this issue's priority, not a second repair. Confirmed the same three facts from the other direction: the health polls at
ansible/playbooks/update-all-nodes.yml:679-689and:1311-1325are followed by no revert task; the snapshot/restore mechanism exists and works on the drift-resolve path (api/code_sync.py:2467-2511called from:3032,_rollback_componentat:2555-2585with seven call sites); and the DB half is fail-closed on both paths (ansible/_shared/tasks/pre_migration_backup.ymlviamigrate_backend_db.yml:29-30).The audit's framing, for whatever it is worth to the fix: data rollback is ahead of the reference work (which reverts the root filesystem but not
/home, and hands the resulting config conflict to the user), while code rollback is behind it on the path operators actually use.Sibling filed on the same path: #17465 — out-of-band mutation of the deployed tree is undetectable, so the builtin-updater-only rule has no detector. Same files, likely one PR.
Analysis:
docs/research/agentic-desktop-os-distribution.md.mrveiss commentedon Sep 27, 2026
Our other rollback surface has the same gap — #17657
Filed #17657:
rollback_deployment(autobot-slm-backend/services/deployment.py:419-440) subtractsrole names from
node.roles, marks the rowROLLED_BACK, and runs no playbook — grepping thewhole function for
playbook/ansiblereturns no hits._perform_auto_rollback(
services/reconciler.py:1546-1571) is the same set-difference.Different subsystem from this issue — that is the whole-machine self-update playbook, this is the
role-deployment service — so it is a sibling rather than a duplicate. Together they mean neither
of our two rollback surfaces performs a rollback, which is worth knowing when either is scheduled.
Also confirming your file-size observation from a second angle, since the comparison this came from
faulted the external reference work for a 3,302-line source file: our
ansible/playbooks/update-all-nodes.ymlis 2,260 lines, and the 600-lineMAX_LINESratchet onlycovers
.py, so the Ansible plane has the same disease and the ratchet structurally cannot see it.Recorded in the umbrella #17656 rather than filed separately, since you already hold this end of it.
docs(research): node-plane comparison findings (#17656)