Skip to content

feat(sync): post-sync actions need a durable record and a node-level plan in the orchestrator #17455

Open
0 of 2 issues completed

Description

@mrveiss

Parent: #17452
Depends on: #17453

Problem

The owner asks that post-sync actions be part of the code-sync process and its logs, rather than a
separate button bar.

An automated path already exists, and that reframes this issue — it is not "build automation",
it is "the automation has no record and no node-level plan". sync_orchestrator.py runs
_run_post_sync_cmd then _restart_systemd_service per role (lines 216-245), in the right order
within a role. What it does not do:

  • leave a durable record. A failure goes to the application logger
    (logger.error("Post-sync command exited %s: %s", ...)) with an 800-character output tail. There
    is no per-action row anyone can read after the fact, so the sync's own history cannot answer
    "what post-sync work ran, and did it succeed".
  • order or deduplicate across roles. Each role syncs independently, so a unit owned by two
    roles (SLM role shape has two sources of truth: role_registry.DEFAULT_ROLES vs autobot-infrastructure/*/manifest.yml #16025) restarts twice here too, and a shared-library install can land after a dependent
    service has already restarted.

So the manual bar is a parallel surface that re-derives the same work, and #17453 is fixing the
ordering on the surface that is not the automated one. That asymmetry is the thing to remove.

Shape

Reuse #17453's ordered_post_sync_plan rather than growing a second ordering policy in the
orchestrator — two node-level plans would be the original defect one level up.

Acceptance criteria

  • Each post-sync action executed by a sync leaves a durable record: what ran, when, exit status,
    output tail.
  • The record is reachable from wherever the sync run is shown, so the manual bar is a fallback
    rather than the only view of this work.
  • The orchestrator orders and deduplicates across roles using the same policy as the endpoint —
    one implementation, imported, not reimplemented.
  • A unit owned by two roles restarts once per sync, asserted by a test.
  • A post-sync failure is visible in the run's record and not only in the application log.

Activity

  1. mrveiss commented on Oct 10, 2026

    @mrveiss
    OwnerAuthor

    Decisions needed (owner): deploy-visible, recorded by the de-fork master session while the owner is away. Surveyed by autobot-ai-ad. Recommendations in brackets:

    1. Restart order on a multi-role node: per role (today) or node-major (all installs → schema → builds → restarts across roles, per _CATEGORY_RANK)? [Node-major, but the self-killing unit (slm-backend, _COMPONENT_SERVICES discovery(deploy): autobot_shared/library restart path has no post-restart health poll or rollback #11496) stays LAST and chromadb restarts before ai-stack. The rule lives once in _post_sync_plan.]
    2. ollama restarts once per node rather than once per role (this issue's AC4). It only exists under node-major. [Yes.]
    3. The orchestrator restart gains sudo -n, failing fast without NOPASSWD sudo instead of waiting out 60 s, with failures recorded rather than swallowed. [Yes, matching the other two restart paths.]
    4. A new post_sync_action_runs table plus a migration: a schema addition, not a rewrite. [Yes.]

    Meanwhile: #18186 (three SSH-restart implementations) is being consolidated with ZERO behaviour change. Each site keeps its current flags and timeouts as explicit parameters, so the -n/timeout unification above becomes a one-line follow-up once decided. The code_sync.py wiring and #17489 wait for the queued redaction branch, which owns that file. code_sync's component step order is unchanged (the recorded #17867 decision).

  2. added
    needs-decisionBlocked on an owner decision; options and a recommendation are on the issue
    on Oct 10, 2026
  3. mrveiss commented on Oct 10, 2026

    @mrveiss
    OwnerAuthor

    Owner decision (2026-10-10): all four yes.

    1. Node-major order (installs, then schema, then builds, then restarts, across roles), with the self-killing slm-backend LAST and chromadb before ai-stack. The rule lives once in _post_sync_plan.
    2. ollama restarts once per node.
    3. The orchestrator restart uses sudo -n and records failures.
    4. A new post_sync_action_runs table plus migration.
  4. removed
    needs-decisionBlocked on an owner decision; options and a recommendation are on the issue
    on Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions