Skip to content

fix: protect presenter launch reconciliation - #361

Merged
therockstorm merged 2 commits into
mainfrom
fix/presenter-reconciliation-races
Aug 10, 2026
Merged

therockstorm merged 2 commits into
mainfrom
fix/presenter-reconciliation-races

Conversation

@therockstorm

Copy link
Copy Markdown
Member

Why

Concurrent Groundcrew CLI processes can reconcile a healthy provisioning run against an empty or stale cmux snapshot while the owning process is preparing or launching it. That can durably fail a healthy run or leave a live agent workspace untracked. This has no direct customer impact; it improves dispatch reliability and prevents operator cleanup work.

Summary

  • Persist the provisioning owner process and a preparing-to-launching generation in each active run.
  • Reconcile only snapshots that still match the durable run state and generation, and keep presenter-less runs alive while their owner process is alive.
  • Close a presented workspace when launch creates it but loses the durable running transition.
  • Add deterministic cross-instance and barrier-controlled concurrency regressions for both race orderings and owner recovery.

Validation

  • node --run verify (8 test files passed; 83 tests passed, 1 skipped)
  • npx vitest run src/run/index.test.ts (17 tests passed)

Notes

Findings: fnd_sig-feat-cli-command-012418b101-_c2c978fbb3 and fnd_sig-feat-cli-command-49a9766219-_65990038f3.

Agent session: codex resume 019fecf3-bf31-73b2-92d8-01afb9524089

🤖 cb-ship:created v1 skill@1.0.2

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cdf9b59e-e3e3-419e-a5cf-6bdabde2de49

📥 Commits

Reviewing files that changed from the base of the PR and between 081227d and 62016fd.

📒 Files selected for processing (2)
  • v2/src/run/index.test.ts
  • v2/src/run/index.ts
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • ClipboardHealth/cbh-core (manual)
🚧 Files skipped from review as they are similar to previous changes (1)
  • v2/src/run/index.ts

📝 Walkthrough

Walkthrough

Run provisioning now records an owner process and phase. Launch reserves ownership before agent startup, validates durable transitions, reconciles snapshots, preserves live provisioning runs, and cleans up workspaces after launch races or failures.

Changes

Provisioning lifecycle

Layer / File(s) Summary
Ownership-aware reconciliation
v2/src/run/index.ts, v2/src/run/index.test.ts
RunRecord stores provisioning ownership and phase. Reconciliation checks snapshots and owner-process liveness. Tests cover live owners, terminated owners, and stale presenter snapshots.
Durable launch transition and cleanup
v2/src/run/index.ts, v2/src/run/index.test.ts
Launch reserves ownership under the store lock, starts the agent after reservation, validates the provisioning generation, and closes the workspace when the transition is lost or launch fails. Tests cover launch races and recovery after presenter failure.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RunModule
  participant Store
  participant Agent
  participant Workspace
  RunModule->>Store: reserve provisioning ownership
  RunModule->>Agent: start agent
  Agent-->>RunModule: return launch result
  RunModule->>Store: verify generation and transition to running
  RunModule->>Workspace: close workspace after failed transition
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: protecting presenter launch reconciliation during concurrent provisioning.
Description check ✅ Passed The description directly explains the reconciliation changes, concurrency protections, failure recovery, and validation results.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/presenter-reconciliation-races

Comment @coderabbitai help to get the list of available commands.

ghost

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
v2/src/run/index.test.ts (2)

522-576: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a regression for a failing agent launch.

This test covers the lost durable transition, where fail() already cleared provisioningOwner. It does not cover launchAgent throwing while the record stays provisioning. That path leaves the reservation in the launching phase, as noted on v2/src/run/index.ts Lines 367-377. Add a test that makes agent startup fail, then asserts that a second launch call does not throw is already launching.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@v2/src/run/index.test.ts` around lines 522 - 576, Add a regression test
alongside the existing lost-transition test that forces agent startup in launch
to throw while the run record remains provisioning, then verifies a subsequent
launch succeeds without an “is already launching” error. Use the existing
RunModule launch setup and failure-injection fixture mechanisms, and assert the
reservation is no longer stuck in the launching phase.

399-424: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the shared presenter fixture setup.

The five affected tests repeat the temporary state root, presenter state files, fake-bin PATH, and environment setup. Use one helper for the common fixture and preserve test-specific environment variables.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@v2/src/run/index.test.ts` around lines 399 - 424, Extract the repeated
temporary presenter fixture setup from the affected tests into a shared helper,
including the state root, presenter state files, fake-bin PATH, and base
environment. Update each test to use the helper while merging its test-specific
environment variables, and preserve the existing RunModule setup and behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@v2/src/run/index.ts`:
- Around line 331-377: In v2/src/run/index.ts lines 331-377, update the
launchAgent catch path to conditionally reset provisioningOwner to the preparing
phase when the runId, provisioning state, and provisioning generation still
match, before closing the workspace and rethrowing. In v2/src/run/index.test.ts
lines 522-576, add coverage that forces agent startup failure and verifies a
subsequent launch does not throw “is already launching”.
- Around line 1185-1196: Update provisioningOwnerIsAlive and the
provisioning-owner identity data it consumes to validate more than PID liveness:
persist and compare the OS boot identifier, plus a platform-specific
process-start identity to detect same-boot PID reuse. Return false when any
recorded identity does not match the current process, while preserving the
existing ESRCH handling. Do not limit startedAt comparison to owners whose
processId equals process.pid.

---

Nitpick comments:
In `@v2/src/run/index.test.ts`:
- Around line 522-576: Add a regression test alongside the existing
lost-transition test that forces agent startup in launch to throw while the run
record remains provisioning, then verifies a subsequent launch succeeds without
an “is already launching” error. Use the existing RunModule launch setup and
failure-injection fixture mechanisms, and assert the reservation is no longer
stuck in the launching phase.
- Around line 399-424: Extract the repeated temporary presenter fixture setup
from the affected tests into a shared helper, including the state root,
presenter state files, fake-bin PATH, and base environment. Update each test to
use the helper while merging its test-specific environment variables, and
preserve the existing RunModule setup and behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 85975874-2a69-4348-b6a2-a93301107f7a

📥 Commits

Reviewing files that changed from the base of the PR and between 74edac9 and 081227d.

📒 Files selected for processing (2)
  • v2/src/run/index.test.ts
  • v2/src/run/index.ts
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • ClipboardHealth/cbh-core (manual)

Comment thread v2/src/run/index.ts
Comment thread v2/src/run/index.ts
@therockstorm

Copy link
Copy Markdown
Member Author

Review-body findings

Mendral reported LGTM and raised no actionable finding.

Agree

  • CodeRabbit: the failed-presenter-launch regression and reservation reset were addressed in 62016fd.

Disagree

  • CodeRabbit: leaving the concurrency-test fixture setup local to each case. The HOME, presenter barrier, and environment combinations are causal inputs to those scenarios; extracting them would hide the distinctions without changing behavior.
a87f27d0bc78d384
801c709beddbf011

🤖 cb-babysit:addressed v1 skill@1.0.4

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The new reservation-release logic at v2/src/run/index.ts:369-384 is correctly guarded by the same CAS predicates (runId, state, generation) used elsewhere, ensuring it's a no-op if the run has already moved on. The test exercises the full retry path through a real RunModule instance. No new issues introduced.

What this PR does

Adds a launch-reservation release in the failure path: when launchAgent or the durable "running" transition fails, the provisioningOwner phase is CAS-reset from "launching" back to "preparing" before closing the presented workspace. This allows a subsequent retry (same or new process) to re-acquire the launch reservation. Includes a regression test verifying end-to-end retry after a presenter open failure.

Tag @mendral-app with feedback or questions. View session

@therockstorm
therockstorm merged commit 3e98f20 into main Aug 10, 2026
8 checks passed
@therockstorm
therockstorm deleted the fix/presenter-reconciliation-races branch August 10, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant