fix(deploy): stop pruning runners whose state cannot be determined

[ss251]

Closes #17566.

Summary

The scheduled runner-workspace pruner now deletes only when it can positively establish one of two safe states:

• no runner or service configuration remains, so the directory is orphaned; or
• the exact systemd unit bound to that runner reports inactive.

Every ambiguous state fails closed and is counted as skipped_undeterminable. This composes with #17535: that merged fix protects GitHub Actions control directories inside _work, while this PR prevents an entire live runner workspace from being misclassified as idle.

Root cause and implementation

The old helper reconstructed a systemd unit by parsing agentName from .runner, substring-searching systemctl list-units, and treating every miss or command failure as an orphan. That approach had three destructive failure modes:

1. a missing, unreadable, or changed .runner shape became “no runner”;
2. runner-1 could resolve to runner-10; and
3. a failed systemctl query became an empty match and therefore permission to prune.

GitHub’s runner service installer already writes the authoritative generated unit name to <runner>/.service. The generated name cannot safely be reconstructed: the runner normalizes spaces and can truncate long names with a random suffix. GitHub’s own troubleshooting documentation recommends reading .service to locate the service.

This PR therefore:

• reads and validates the exact .service binding instead of guessing from agentName;
• skips configured runners with missing, unreadable, or malformed bindings;
• queries that exact unit and prunes only on the literal inactive state;
• skips active, failed, activating/deactivating, unknown, empty, and command-failure states;
• preserves genuine orphan cleanup when neither .runner nor .service remains; and
• reports active, undeterminable, pruned, and failed counts separately.

Primary references: GitHub runner service-name generation, systemd service installer, and GitHub’s .service guidance.

Risk

The behavior is asymmetric in the safe direction: uncertainty converts a deletion into retention. A permanently broken binding can consume disk, but the summary exposes skipped_undeterminable so operators can repair it. A healthy inactive runner and a configuration-free orphan remain reclaimable.

Validation

Exact head 115a04ed0fbc43e5bd526fd8fc4878415287b1ad, rebased directly on merged #17535:

• bash packages/deploy/runner-workspace-cleanup/smoke-test.sh — passed against the real prune tool.
• shellcheck on the helper and smoke suite — passed.
• bun run verify — 581/581 build/typecheck/lint tasks and every repository audit passed.
• Mutation: allowing a configured runner without .service to reach the orphan branch fails with configured runner without .service binding was pruned.
• Mutation: ignoring the exact binding and selecting runner-10 fails with busy runner.1 was pruned instead of using its exact .service binding.
• Restoring the implementation returns the full smoke suite to green.

The smoke suite also proves #17535’s runner-command-file fixture survives, missing/malformed bindings survive, unknown and failed unit states survive, a dotted active runner is not confused with an idle sibling, and a genuine orphan is reclaimed.

Evidence

• N/A - host-local shell infrastructure has no rendered user interface.

• N/A - host-local shell infrastructure has no rendered user interface.

• N/A - there is no interactive UI flow for this systemd timer helper.

• N/A - the host-local timer does not execute through an application backend service.

• N/A - no frontend code or client transport is involved.

• N/A - no model, prompt, provider, action, or evaluator behavior changes.

• Exact-head smoke, shellcheck, full verify, both red mutations, and restored green run

Contribution provenance

• AI assistance: yes

• Model(s) used: anthropic/claude-fable-5, openai/gpt-5

• Client / agent tooling: Claude Code; Codex desktop

• Skill revision: elizaOS/eliza@e37924817dd457f3c5cc1547531c7054336a3740:packages/skills/skills/contribute-to-eliza

• Attribution status: self-reported

[sharkwon]

CLAIMING REVIEW: fix(deploy) runner-workspace-cleanup — independent review of the fail-open pruner fix and exact unit-suffix matching (Closes #17566), including reproduction of §11/§12 smoke tests.

AI provider/model: custom / ds/deepseek-v4-pro-max
Client / agent tooling: Hermes
Contribution skill revision: e379248:packages/skills/skills/contribute-to-eliza
Attribution status: self-reported
— [sharkwon]

[sharkwon]

Independent review — @sharkwon

Reviewed head 833d2f882743acefeeb3b692004ae324e0ca1816 (1 commit, zero-behind develop @ ba62c0234). Scope: packages/deploy/runner-workspace-cleanup — 2 files, +123/−8. Both changed files are executable shell scripts; no lockfile, package.json, lifecycle hooks, symlinks, generated binaries, or CI workflow changes — nothing untrusted to execute.

Reproduced independently (disposable sandbox)

Ran the PR's smoke test from the PR head inside an isolated bwrap/userns sandbox (fresh HOME, GIT_CONFIG_*=/dev/null, no credentials, network denied, stage tree without .git):

• Green (PR head): runner-workspace-cleanup smoke OK (behavioral) — exit 0.
• Mutation 1 — revert helper to origin/develop (fail-open defect): smoke test fails with smoke failed: unreadable .runner was pruned — the helper cannot prove that runner is idle and must refuse — exit 1. §11 is genuine regression coverage.
• Mutation 2 — restore old grep -F substring unit match (Defect 2): smoke test fails with smoke failed: BUSY runner-1 was pruned — its unit resolved to the idle runner-10 by substring match — exit 1. §12 is genuine regression coverage.

I also opened the PR's attached evidence: red log (mutation proof failing), green log (passing suite), sweep + filesystem-state log. They match the described runs. Video link resolves (trimmed recording of the same sequence).

Assessment

The change is correct and the tests are meaningful:

• Defect 1 (fail-open): The -e/-r split plus the undeterminable counter correctly distinguishes absent (prunable orphan, #15398 preserved) from present-but-unreadable/unparseable (skipped, reported, counted). The asymmetry is in the safe direction — a prune becomes a skip, never a new deletion.
• Defect 2 (unit mismatch): awk -v want="$agent_name" -F. '$NF == want {print; exit}' after stripping actions.runner. prefix and .service suffix is an exact suffix match, closing the runner-1 ↔ runner-10 collision. Stub ordering in §12 deliberately lists the colliding unit first, which is the correct adversarial shape.
• The smoke test drives the real helper and the real prune tool (only systemctl stubbed, consistent with the pre-existing §10 pattern). --allow-active stays on the prune path only after the per-runner inactivity proof, which is the design's stated invariant.

Non-blocking notes

1. chmod 000 fixture is weaker under root. The shipped unit runs as root with CapabilityBoundingSet=CAP_DAC_OVERRIDE CAP_DAC_READ_SEARCH, so on a real runner host an unreadable-permission .runner is in fact still readable by the helper — the unreadable-runner fixture would not reproduce there as written (it does reproduce on CI runners and non-root shells). The unparseable fixture ({"poolId": 3}) is the root-relevant case and is solid. Worth a comment in the test that the mode-000 case is permission-level only.
2. Develop PR Gate failure is CI infrastructure, not this PR. The gate timed out waiting for 7 required checks that never started (no GitHub Actions check run from .github/workflows/develop-pr.yml etc.) — the same 40m failure signature appears on #17578 and #17577; the older #17573 passed the gate. No check result contradicts this diff.
3. bun run verify not run is correctly scoped: neither changed file is in the TS project graph. Smoke test is the applicable gate.

No blocking findings. This fix and #17544 are complementary as the author states — both should land.

---

AI provider/model: custom / ds/deepseek-v4-pro-max
Client / agent tooling: Hermes
Contribution skill revision: e379248:packages/skills/skills/contribute-to-eliza
Attribution status: self-reported
— [sharkwon]