Initiative: Safe Release Strategy for Agentic Workflows (versioning · rings · canary)

[don-petry]

Initiative: Safe Release Strategy for Agentic Workflows

Theme: Agentic Framework · Initiative: Release Strategy · Status: Proposed (for review)

Reimagine how we build, test, release, validate, and roll back our core agentic workflows
(dev-lead, pr-review) and their delivery path — so new versions can be proven healthy
before the prior known-good version is retired. Eliminates the self-hosting circular
dependency and the "instantly live across the whole fleet" risk.

📄 Full analysis: docs/initiatives/agentic-release-strategy.md (problem statement, real
metrics, success criteria, options analysis, recommendation) — see linked PR.

The problem (two compounding root causes)

• A — Self-hosting circular dependency. dev-lead/pr-review build, review, and merge changes
  to themselves. A broken in-flight version blocks the fix for its own breakage (PR Review Agent never reviews at green CI — skips on ci-pending with no CI-completion re-trigger (clean PRs stuck at REVIEW_REQUIRED) #463, dev-lead: lone cancelled check cycles 30-minute retries forever — nothing re-triggers the cancelled workflow #466, feat(pr-reviewer): re-implement 7 safety checks against current architecture #305).
• B — No version boundary. Every caller references the reusables at @main; there are
  0 tags / 0 releases. A merge to main is instantly live across self-review duty and all
  four consumer repos, with no canary, no health gate, no rollback. Deploy is a PUT /contents
  clobber of main across all consumers at once.

Evidence (2026-06-08)

• 48 automated failure-report issues since 2026-05-06 (~daily); 20 still open, 11 consecutive
  unresolved days.
• PR Review Agent: 0 successes in last 20 runs (all cancelled); 34% overall cancellation rate.
• 0 git tags, 0 releases; all callers pin @main.
• Self-repair churn peaked at 123 commits/week.

Success criteria (must-haves in bold)

• SC1 A known-good version is always referenceable & serving production duty.
• SC2 The circular dependency is eliminated — a broken in-dev version cannot block its own fix PR.
• SC3 Blast radius is staged ring-by-ring, never fleet-wide-instant.
• SC4 One-action rollback in < 5 min (pointer flip / revert), no manual file surgery.
• SC5 New versions validated in ring 0 before any consumer sees them.
• SC6 Failure-report backlog < 5 (from 20) and cancellation rate < 10% (from 34%) within 60 days.
• SC7 Innovation velocity preserved (continuous shipping to next).
• SC8 Every ring/consumer's pinned version is reportable.

Recommended approach — Option C: versioned releases + concentric rings + health-gated promotion

GitHub-native only (tags/releases, @ref pinning, Environments, matrix staging). Ring 0 =
.github-private self-host (dogfood). Production self-review/dev duty stays pinned to stable while
new versions are exercised on next — that is what breaks the circular dependency.

• Phase 1 (version boundary): versioning scheme → cut first release tag → pin all callers
  (consumers + self) off @main → release runbook.
• Phase 2 (rings + automation): stable/next channel pointers → define rings 0/1/2 → replace
  clobber-deploy with ring-staged health-gated promotion → one-action rollback → per-version
  observability → SC2 game-day regression test.

Constraints (confirmed)

Scope: core agentic + delivery. Infra: GitHub-native only (no new repos / external infra). Canary:
.github-private self-host.

Child workstreams

Tracked as sub-issues on this epic (see task list / linked items).