Add black-box e2e tests driving the nix-built binary

[srid]

odu now has end-to-end tests that exercise the real CLI as a black box — building the binary with nix build, running it against a throwaway fixture repo on a localhost lane, and asserting on its --progress json stream and process exit code. Until now the only integration coverage was the in-process loopback suite (src/odu.test.ts), which stubs the transport with an in-memory stream pair. These tests cover the seams that suite fakes.

What it proves end-to-end

just DAG ingest ─▶ scheduling ─▶ local lane spawn (odu-runner) ─▶ NDJSON projection ─▶ exit code

The suite spawns the actual odu binary — nothing is imported from src/, so the contract under test is the binary's observable behavior, not its internals. Two fixtures drive the two outcomes that matter:

• a passing DAG → every node goes green, odu run exits 0
• a failing DAG → a node fails, that failure surfaces as status: "failed" / exit_code: 1 and the process exits 1

Plus a smoke test for odu dump (resolves the pipeline without a live socket) and a shape check on the progress events.

The fixture is a flake, on purpose

A local odu run resolves its lane runner by evaluating <repo>#packages.<system>.odu-runner.drvPath — even for a localhost lane (the realise is a local no-op copy, but the drvPath lookup still happens). So any repo that runs odu locally must expose odu-runner in its flake. Each fixture is therefore a tiny flake that re-exports odu-runner from the checkout under test, so it runs the exact runner the harness just built (a Nix cache hit). The leaf recipes are pure shell — the fixture's own "CI" is trivial so the test exercises odu's machinery, not a toolchain.

Deliberate tradeoffs (documented in tests/e2e/README.md)

Choice	Why
Harness builds the binary itself	Self-contained, order-independent; cold-cache build on first run, cache hits after
Assert on --progress json (NDJSON), not the TTY dashboard	Clean, parseable contract; dashboard is a separate path (a follow-up)
Local-only, --no-strict	No ssh/remote lanes yet; localhost against the live tree
Black-box (no src/ imports)	Internal refactors don't ripple into the tests

Wired in as a new e2e step in ci/mod.just (and a top-level just e2e). Follow-ups — TTY dashboard / attach, a real ssh-to-localhost transport, the MCP face, and status/logs against a live socket — are listed in the README.

The hickey+lowy structural review's first pass wanted the duplicated ProgressEvent wire type deduplicated into src/; cross-validation reversed that — the duplication is a load-bearing black-box boundary, and importing it would make the test white-box and hide the very wire-format regressions it exists to catch. See the analysis comment below.

Generated by /do on Claude Code (model claude-opus-4-8).

[srid]

Hickey/Lowy Analysis

Both lenses ran in parallel on the feature commit, then a cross-validation pass audited each lens against the other's recommendations. The cross-validation reversed three first-pass findings — a good example of why it exists.

#	Lens	Finding	Disposition
1	Hickey + Lowy	ProgressEvent duplicated in harness.ts rather than shared with src/coordinator/display.ts	⚠️ No-op
2	Hickey + Lowy	harness.ts bundles ~5 concerns / 3 volatility axes — split into modules	⚠️ No-op
3	Lowy	dump test calls execFileSync directly, bypassing the harness CLI receptacle	⚠️ No-op
4	Hickey	Fixture cleanup via module-level array + afterAll sweep	Fixed in this PR
5	Hickey	cleanup() swallows errors silently	Fixed in this PR

Why #1–#3 became No-ops (cross-validation)

• odu: a CI runner you attach to — graduated from kolu #1 (the headline): First pass (both lenses) wanted the wire type deduplicated into src/. The cross-validators reversed it: the local copy is a load-bearing black-box projection schema — the test deserializes exactly this shape from the real binary's output. Importing the type would make the test white-box and hide the wire-format regressions the suite exists to catch. Lowy also flagged that surface.ts (Hickey's preferred home) is the wrong receptacle — it would add a third volatility axis there. Resolution: keep the duplication, add a comment locking in the intent so nobody "fixes" it later.
• Ship the /ci skill: odu documents itself the way justci did #2: Hickey's cross-validation called the split over-organization for a ~150-line single-consumer helper whose stages co-change; per-function JSDoc already marks the axes.
• feat(mcp): the agent face — odu mcp server (tools + resources) #3: oduRun (NDJSON stream parse) and the dump call (raw one-shot JSON) are genuinely different invocation shapes; unifying them relocates complexity to a discriminator.

Hickey rationale

The one real structural defect was the ProgressEvent fragmentation — but cross-validation established that this particular duplication is the correct black-box boundary, not accidental complexity. The actionable wins were both small and local: co-locating fixture teardown with its creation (onTestFinished) and making cleanup failures observable instead of silent.

Lowy rationale

The big boundaries are right: vitest.e2e.config.ts encapsulates test-runner config volatility, the fixtures are pure data, and the black-box policy is a sound change-encapsulation boundary. The only volatility error worth flagging was the proposed destination for the (rejected) ProgressEvent move — surface.ts already carries surface-protocol + status-vocabulary axes, so the wire-output contract doesn't belong there.

[srid]

/do results

Step	Status	Duration	Verification
sync	✓	1s	git fetch ok; forge=github
research	✓	6m 45s	Mapped odu run local-lane internals; key discovery: resolveDrvPath runs even for localhost, so each fixture must be a flake exposing odu-runner
branch	✓	~0s	On feature branch e2e
implement	✓	~0s*	tests/e2e/ suite + fixtures + vitest.e2e config + test:e2e + e2e CI step; 4/4 green
check	✓	6s	pnpm typecheck clean (now covers tests/)
docs	✓	~0s	README Developing block + top-level just e2e; tests/e2e/README.md
fmt	—	0s	skipped — no format command configured
commit	✓	12s	feature commit 2c8a041 pushed
hickey+lowy	✓	9m 25s	parallel review + cross-validation; 3 first-pass Fixes overturned to No-op, 2 real Fixes committed
police	✓	~0s*	3 passes; dry-rule + no-silent-swallow fixed, 4 elegance fixes, 2 documented No-ops
test	✓	28s	test:e2e 4/4, test:unit 80/80
create-pr	✓	48s	draft PR #20 + hickey/lowy ledger
ci	✓	~14m*	GitHub Actions green at HEAD e9c074f on both platforms (ci::e2e@x86_64-linux + ci::e2e@aarch64-darwin)
evidence	—	0s	skipped — not opted in
Total		~43m

_{*Some per-step durations are approximate — a few do-results start/end bookends were recorded back-to-back, and the ci wall-clock (~14m, cold nix on macOS) landed outside its recorded window.}

Slowest step: hickey+lowy (9m 25s)

Optimization suggestions

• hickey+lowy (9m 25s) dominated — the cross-validation pass (4 extra sub-agents) is what earned its keep here, reversing the headline ProgressEvent-dedup finding. Worth it on a new-abstraction PR; for a trivial diff, --minimal skips this entirely.
• research (6m 45s) — the whole design hinged on one fact: a local odu run still does nix eval <repo>#…odu-runner.drvPath. Pre-reading src/coordinator/run.ts (the startLane/resolveDrvPath block) and @kolu/surface-nix-host's getHostSession before invoking /do would cut this in half.
• ci cold-cache (~14m, macOS) — the e2e step's nix build is the cost; it's a cache hit on warm runners. For re-runs after a fix, --from ci-only skips straight to it.
• The e2e suite's first run pays a cold nix build .#odu; locally, just e2e is fast once the store is warm.

Workflow completed.