land: pipelining - prepare batch K+1 on batch K's top while K publishes

[thejackshelton]

What changed: pipelined landing. While batch K publishes, a builder process prepares batch K+1 on K's proven top. Publishing is mostly CI and review waits; the builder's preparation covers admission, positions, regen, devices, the full test and the bisect. When K has landed, K+1 is usually proven already and starts publishing at once.

Design

prepareRound in land-lib.ts. The admission, build, chain verification, top proof and bisect move out of runBatches into prepareRound. It publishes nothing. The driver's own round reports each result as it happens, exactly as before; a builder's round returns its results instead. The sequential path behaves as before; all 87 existing land.test.ts tests pass unchanged.

When the next round starts. runBatches starts preparing the next round (NextRound.start) only when the current batch's top passed, so its whole chain is proven. It hands over three things:

• the base: K's top position;
• a snapshot of the queue;
• K's PRs as earlier, so a child of a K PR is admitted.

Using it: only if all of K landed. Master then has exactly the tree the next batch was built on. Position 1's previous position is K's top, which is in master with master's tree, so every merge gate and the push check hold as for any position. The driver then:

• takes the consumed PRs off the front of the queue, checking they are the front;
• runs verifyChain on the adopted chain itself;
• reports the builder's results in order;
• adds the proven positions to the run's proved list;
• publishes as usual.

Throwing it away otherwise. If K stopped part-way (a failed publish, a culprit, requeued PRs), or a stop is requested, the prepared round is discarded and its builder stopped. Nothing from it is reported: no labels, no comments, and no ejection. A merge conflict there could have come from a K PR that never landed. Its PRs are prepared again by the driver on the new master. A stop request is now read once and holds for the run (reading it removes STOP_FILE).

No master proof in the builder. A bisect in the builder that lands on position 1 does not prove master. Its base is K's proven top (baseProven).

The builder is land.ts with LAND_ROLE=builder:

• Worktree: LAND_WORKTREE_NEXT, default /tmp/dragon-land-next. It is reset at start, including a stale index.lock.
• Input and output: it reads next-input.json and writes next-output.json atomically in the run directory. The output is the round, or { fatal }.
• Writes nothing shared: no status, no labels, no comments.
• Logs: to the driver's log with a [next] prefix, and to /tmp/land-next.log.
• Lifetime: it runs in the driver's process group, so a supervisor interrupt kills it with the driver. It stops itself (exitIfOrphaned) if the driver dies.

How the driver handles the builder:

• Waiting. The driver polls for the output file while the builder is running, and does not count a zombie as running, because the synchronous driver never reaps it.
• Stopping. It stops the builder's whole process tree (SIGTERM, up to 30 s, then SIGKILL) and releases the builder's quiet request and priority.
• No usable output. If the builder dies without output or writes malformed output, the driver prepares the round itself.
• Builder Fatal. A Fatal in the builder becomes a Fatal of the run.
• Parallel fetches. Fetching from two processes can contend for a ref lock. "cannot lock ref" and "Unable to create *.lock" are now transient errors, retried with backoff.

LAND_PIPELINE=0 turns pipelining off. The worktree lists exclude both driver worktrees.

Outside the spec: AGENTS.md step 7 now describes pipelining.

What passed

• pnpm typecheck passed.
• vitest run packages/parity/test/land.test.ts passed: 92 tests, 5 of them new, with a fake builder that runs prepareRound with the same fakes:
  • the next batch is built and proved on the top before the first publish, then adopted, and the driver never builds those PRs itself;
  • it is thrown away when a publish fails, its results go unreported, and the driver reports a rejection once;
  • nothing is prepared ahead of a failing top, and a stop request discards the prepared batch;
  • a builder's Fatal stops the run, and a builder that ended with nothing makes the driver prepare the batch itself;
  • serializePrepared and parsePrepared round-trip exactly and refuse malformed rounds.
• Smoke test of the builder process: on master with an empty queue it wrote an empty round, and with a bad base it wrote { fatal }. It exited 0 both times.
• Not run: a live pipelined landing. The driver proves it on its next queue.

Review round 1 (precomputed review of db85440: 1 Medium, 2 Low)

• Medium: the builder outlived a failed publish. When a publish fails, runBatches now stops the builder immediately, before the proof of the tree master rests on. The builder's tests can no longer compete with that proof or starve its quiet rerun.
  • New test: the fixedBy scenario. In it, 'next cancel' comes after the failed publish and before the resting-tree proof, exactly once.
  • The test fails without the fix.
• Low: builder infrastructure errors. A failed worktree add, a git error or a full disk in the builder no longer becomes a Fatal. The builder logs it, writes no output and exits 1, and the driver prepares the batch itself. Only a real Fatal, such as a chain that is not what it claims, still stops the run.
  • Smoke test: with an impossible worktree path, the builder wrote no output and exited 1.
• Low: stopping the builder by its own process group.
  • The builder is spawned detached, in its own process group. It is stopped by signalling that group: SIGTERM, up to 30 s while any non-zombie member lives, then SIGKILL. Late-spawned vitest workers go with it.
  • Its pid and start time are kept in the run directory as builder.pid. A leader that has since become another process is left alone.
  • The supervisor's cleanup after a driver death stops the builder too, because an interrupt of the driver's group does not reach it.

What passed:

• pnpm typecheck passed.
• land.test.ts passed: 93 tests.

🤖 Generated with Claude Code