Skip to content

feat(scripts): --base runs a second live-test server at another ref - #558

Merged
Jacksondr5 merged 8 commits into
j5/mainfrom
j5code/live-test-base
Oct 12, 2026
Merged

Jacksondr5 merged 8 commits into
j5/mainfrom
j5code/live-test-base

Conversation

@Jacksondr5

@Jacksondr5 Jacksondr5 commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Problem

A UI change needs before and after screenshots on the same data. Today an agent builds a second checkout and a second seeded server by hand to get the "before". Part of #546 (stage 3, fourth of five stacked PRs; based on #556).

What changed

A start or rebuild takes --base <ref>:

node scripts/j5/setup-agent-live-test.ts --base origin/j5/main

It starts the usual server from this checkout (the head) and a second one from the base ref.

  • The base is its own checkout. The script adds a git worktree of this repository at the ref, detached, at .j5code/live-test/base/, and installs it with pnpm install --frozen-lockfile.
  • The base runs its own copy of this script. A linked worktree already gets its own derived ports, its own .j5code/live-test, its own generator and its own process record. So the head script adds no second code path for starting, stopping or proving a server: status, restart and stop call the base checkout's script with the same command and print what it says. A home built by the head's generator is never opened by the base's code.
  • One clock for both builds. The generator reads its clock from J5_LIVE_TEST_NOW when it is set. The script chooses the clock once, records it in the manifest, and gives it to the base on every start, so a retry dates the base the same. Thread ids are still random on each side.
  • The output says when the dates will not match. A base from before this PR ignores the clock. That is every base today, origin/j5/main included, until this merges. The block then says the base's threads are dated from its own build.
  • The checkout is verified on every command, never reset. <root>/base must be this repository checked out at the commit the build recorded; otherwise the script refuses and names rebuild. On stop the check comes after the base server has stopped. rebuild skips it, because replacing the build is the way out of a moved checkout.
  • The output says what the checkout is. It is a git worktree of this repository, detached with no branch, and rebuild with no flag removes it when the before and after work is done.
  • A base without the script is refused, by name, before anything is built or deleted: a ref older than feat(scripts): one command sets up a live-test server in a worktree #551 has no generator. A ref with shell characters is refused too, because the ref is printed inside commands.
  • The base is prepared in resumable foreground steps. Checkout, install, then start the base's server. Each step runs in the foreground, in a process group of its own, and leaves its result on disk: the checkout at the recorded commit, a marker written last by the install, the base's own server. A run does as many steps as fit in the same time budget as the head. If the time runs out, the step's whole group is ended and waited for, so nothing it started (an install, a build) is left writing; the command then lists each step as done or not done, exits non-zero, and says to run the same command again, which picks up at the first step that is not done. A base build that was cut off part-way is rebuilt. status lists the steps as done, not done or failed from what is on disk. stop and rebuild have nothing hidden to race with. The head server boots in its own process while the base's steps run.
  • A switch of mode removes the checkout through git. The base is a registered git worktree, created detached so no branch is made. stop leaves it. rebuild stops the base server, then runs git worktree remove on it, not forced, before deleting the directory. If git will not remove it, the script refuses with git's reason and deletes nothing else.
  • --base goes with the default build only; it is refused with --fresh, --peer or --for-dev.

Live testing

On 2026-10-11, from this branch's worktree.

Exercised:

  • rebuild --base origin/j5/main (631025be1b, a base without the clock support): 83 s for the head build, the checkout, its install, its build and both starts. Head on web port 7594, base on 5829, each printing its own block.
  • rebuild --base HEAD (a base with the clock support): the 15 replayed threads have identical created_at values in the two homes.
  • status and stop report both servers; the base's port is free after stop. restart: 13 s.
  • --base 276950b5f1 and --base origin/j5/main~1: refused as having no script, with the existing build left in place.
  • I moved the base checkout to another commit by hand: status refused, named both commits and said it had not changed the checkout.
  • status with everything up lists the three steps as done, then the base's own block. After a stop it lists server as not done and names the command to continue.
  • Resume: with the install marker removed and the base stopped, the same start command redid the install and started the base in 15 s.
  • A base build cut off part-way, simulated by removing the base home's manifest: the same start command rebuilt it and started the base in 42 s.
  • A checkout moved to another commit shows checkout failed in status, with both commits named and the checkout left as it was.
  • rebuild with no flag afterwards: the base checkout is gone from git worktree list and from disk.

Known limits of the shared clock, both measured by the tester on a clock-aware base: all 15 replayed threads and their messages have identical creation times in the two homes, but (1) the two delivery-seed threads are stamped by the seed with its own time, and (2) the two turn_interrupt threads get a different updated_at on each side, because each server finishes their checkpoint on its first start. Both show up as different activity ages in Fleet and the sidebar for those four threads.

Not exercised live:

  • Opening both servers in a browser and comparing a screen. I checked the databases, not the pixels.
  • A base whose install fails, or whose server does not start.
  • The time budget actually running out mid-step on a real build. A test covers the mechanism: a command whose child starts a grandchild is cut at its deadline and neither survives. On this host the whole start took 79 s in one run, inside the 100-second budget, and no single step came near it (install about 20 s; the base's own build and start about 35 s). The resume path was exercised by removing a step's result by hand, not by a timeout.
  • A base ref that has moved since the build (the refusal reads the recorded commit; not triggered live).

Checklist

  • One concern: the description has no "also"
  • Tests cover the changed behavior (backend changes ship with focused tests)
  • UI changes: before/after screenshots above, and a video for motion or interaction (no UI change)
  • Upstream-owned files: each one is recorded in FORK.md (case text and file-table row) in this PR (none edited)
  • Upstream product: any change to what upstream's product does has a human decision linked above and a register entry in docs/j5/product/upstream.md (none)
  • Surfaces: entry points, clients, providers, contracts, reverse states, connection modes (see AGENTS.md). stop stops both; rebuild removes the checkout.
  • Docs: definitions under docs/j5/product/ and user docs rewritten where this changes them (the skill text is in docs(j5): agents live-test with one setup script, the Browser panel and a testing skill #549)

Tests: vp test run scripts/j5/setup-agent-live-test.test.ts apps/server/src/j5/test-support/liveTestHome.test.ts (49 pass): --base parsing with its ref and the combinations refused, the containment check on the base checkout, and the generator dating its threads from the given clock. Typecheck and lint are clean on the touched files.

Built with Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
j5-code Ready Ready Preview Oct 12, 2026 5:34am UTC

Request Review

@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from c39ae0a to 7c919e0 Compare October 12, 2026 04:05
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from a8483d3 to b9e4554 Compare October 12, 2026 04:05
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 7c919e0 to 88d81ed Compare October 12, 2026 04:06
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from b9e4554 to 8d683f2 Compare October 12, 2026 04:06
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 88d81ed to 19da1ed Compare October 12, 2026 04:09
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from 8d683f2 to ded7b66 Compare October 12, 2026 04:09
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 19da1ed to 7140218 Compare October 12, 2026 04:16
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from ded7b66 to d3dde9b Compare October 12, 2026 04:16
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 7140218 to 561c2ed Compare October 12, 2026 04:24
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from d3dde9b to 794867d Compare October 12, 2026 04:24
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from 794867d to 2a9db07 Compare October 12, 2026 04:29
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…tries and bounds its start

Review fixes for #558:
- Before the base's script is run, the checkout must be this repository at
  the commit the build recorded. It is never reset. A failing base status
  now fails the command.
- The clock is recorded in the manifest and given to the base on every start,
  and the output says when the base predates the clock and cannot honour it.
- The base's checkout, install and start run in their own process. A start
  waits for it within the one time budget and otherwise returns, saying it is
  still being prepared.
- A ref with shell characters is refused, since the ref is printed inside
  commands.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 3cdf4ad to 1b935fd Compare October 12, 2026 04:43
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…tries and bounds its start

Review fixes for #558:
- Before the base's script is run, the checkout must be this repository at
  the commit the build recorded. It is never reset. A failing base status
  now fails the command.
- The clock is recorded in the manifest and given to the base on every start,
  and the output says when the base predates the clock and cannot honour it.
- The base's checkout, install and start run in their own process. A start
  waits for it within the one time budget and otherwise returns, saying it is
  still being prepared.
- A ref with shell characters is refused, since the ref is printed inside
  commands.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from e6c7a15 to 57ebd01 Compare October 12, 2026 04:43
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 1b935fd to a66e0c1 Compare October 12, 2026 04:46
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…tries and bounds its start

Review fixes for #558:
- Before the base's script is run, the checkout must be this repository at
  the commit the build recorded. It is never reset. A failing base status
  now fails the command.
- The clock is recorded in the manifest and given to the base on every start,
  and the output says when the base predates the clock and cannot honour it.
- The base's checkout, install and start run in their own process. A start
  waits for it within the one time budget and otherwise returns, saying it is
  still being prepared.
- A ref with shell characters is refused, since the ref is printed inside
  commands.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from 57ebd01 to 0ee4c5e Compare October 12, 2026 04:46
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-for-dev branch from 8f2afe7 to 4bd7b5d Compare October 12, 2026 05:18
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…tries and bounds its start

Review fixes for #558:
- Before the base's script is run, the checkout must be this repository at
  the commit the build recorded. It is never reset. A failing base status
  now fails the command.
- The clock is recorded in the manifest and given to the base on every start,
  and the output says when the base predates the clock and cannot honour it.
- The base's checkout, install and start run in their own process. A start
  waits for it within the one time budget and otherwise returns, saying it is
  still being prepared.
- A ref with shell characters is refused, since the ref is printed inside
  commands.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…d knows about

Review fix for #558. The base's checkout, install and start ran in a detached
process nothing recorded, so a rebuild could delete the checkout under a
running install, and a second start ran a second install beside the first.
Its pid and start time are now recorded as a server's are. A start joins a
preparation that is running, status reports it as running or failed, and
stop and rebuild stop it on the same proof before touching the checkout.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from 621ccfb to 58b84d1 Compare October 12, 2026 05:18
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from 58b84d1 to ccc20f2 Compare October 12, 2026 05:20
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…tries and bounds its start

Review fixes for #558:
- Before the base's script is run, the checkout must be this repository at
  the commit the build recorded. It is never reset. A failing base status
  now fails the command.
- The clock is recorded in the manifest and given to the base on every start,
  and the output says when the base predates the clock and cannot honour it.
- The base's checkout, install and start run in their own process. A start
  waits for it within the one time budget and otherwise returns, saying it is
  still being prepared.
- A ref with shell characters is refused, since the ref is printed inside
  commands.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…d knows about

Review fix for #558. The base's checkout, install and start ran in a detached
process nothing recorded, so a rebuild could delete the checkout under a
running install, and a second start ran a second install beside the first.
Its pid and start time are now recorded as a server's are. A start joins a
preparation that is running, status reports it as running or failed, and
stop and rebuild stop it on the same proof before touching the checkout.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jacksondr5 added a commit that referenced this pull request Oct 12, 2026
…tarted with it

Review fixes for #558:
- Each step runs in its own process group, awaited in the foreground. At the
  deadline the whole group is ended and waited for, so an install or a build
  the step started cannot go on writing after the command has returned.
- A base build cut off part-way leaves a home its own script will not build
  over, so the server step rebuilds it and the same command recovers.
- When time has run out, the steps are reported from disk without asking the
  base's server.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from j5code/live-test-for-dev to j5/main October 12, 2026 05:31
Jacksondr5 and others added 8 commits October 12, 2026 01:33
A start or rebuild with --base <ref> adds a git worktree of this repository
at that ref under .j5code/live-test, installs it, and runs that checkout's
own copy of this script, so the base gets its own ports, home, generator and
process record. Both builds are given one clock, which the generator now
reads from J5_LIVE_TEST_NOW, so their replayed threads are dated alike. A ref
without the script is refused before anything is deleted.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A switch of mode removes the base with git worktree remove, so no stale
worktree entry is left, and refuses with git's reason if git will not.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tries and bounds its start

Review fixes for #558:
- Before the base's script is run, the checkout must be this repository at
  the commit the build recorded. It is never reset. A failing base status
  now fails the command.
- The clock is recorded in the manifest and given to the base on every start,
  and the output says when the base predates the clock and cannot honour it.
- The base's checkout, install and start run in their own process. A start
  waits for it within the one time budget and otherwise returns, saying it is
  still being prepared.
- A ref with shell characters is refused, since the ref is printed inside
  commands.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…remove it

The base checkout is verified on every command; on stop the check comes
after the base server has stopped, and a rebuild skips it because it replaces
the build. The output says the checkout is a worktree of this repository and
which command removes it when the before and after work is done.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d knows about

Review fix for #558. The base's checkout, install and start ran in a detached
process nothing recorded, so a rebuild could delete the checkout under a
running install, and a second start ran a second install beside the first.
Its pid and start time are now recorded as a server's are. A start joins a
preparation that is running, status reports it as running or failed, and
stop and rebuild stop it on the same proof before touching the checkout.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The base's checkout, install and start no longer run in a detached process.
Each run of the command does the steps in the foreground, as far as fits in
its time budget, and leaves each step's result on disk. A run that is out of
time stops, says which steps are done and which are left, and exits non-zero;
the same command picks up at the first step that is not done. status reports
each step as done, not done or failed from what is on disk, and stop and
rebuild have nothing hidden to race with.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tarted with it

Review fixes for #558:
- Each step runs in its own process group, awaited in the foreground. At the
  deadline the whole group is ended and waited for, so an install or a build
  the step started cannot go on writing after the command has returned.
- A base build cut off part-way leaves a home its own script will not build
  over, so the server step rebuilds it and the same command recovers.
- When time has run out, the steps are reported from disk without asking the
  base's server.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hard kill was armed only after the step's command had exited, so a
command that ignored SIGTERM held the script for ever. It is now armed when
the deadline passes. If the script itself is told to stop while a step runs,
it ends the step's group first.

Part of #546.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jacksondr5
Jacksondr5 force-pushed the j5code/live-test-base branch from e0c7c31 to 43b4d78 Compare October 12, 2026 05:33
@Jacksondr5
Jacksondr5 marked this pull request as ready for review October 12, 2026 05:44
@Jacksondr5
Jacksondr5 merged commit 7d56979 into j5/main Oct 12, 2026
33 checks passed
@Jacksondr5
Jacksondr5 deleted the j5code/live-test-base branch October 12, 2026 05:44
@coderabbitai

coderabbitai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 4 billable files and costs up to $1.00.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 48 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available. Your 51 included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: ea806afa-0d49-447f-81f2-9e07dde89fd5

📥 Commits

Reviewing files that changed from the base of the PR and between 64e0ccf and 43b4d78.


📒 Files selected for processing (4)
  • apps/server/src/j5/test-support/liveTestHome.test.ts
  • apps/server/src/j5/test-support/liveTestHome.ts
  • scripts/j5/setup-agent-live-test.test.ts
  • scripts/j5/setup-agent-live-test.ts

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

This branch was successfully deployed

1 active deployment
Preview — 43b4d78a Deployed Oct 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant