Skip to content

Lanes verdict at the edge: lanes.json no longer invalidates every regen step - #153

Merged
thejackshelton merged 51 commits into
masterfrom
native-lanes-at-edge
Oct 5, 2026
Merged

thejackshelton merged 51 commits into
masterfrom
native-lanes-at-edge

Conversation

@thejackshelton

Copy link
Copy Markdown
Contributor

Why

packages/dragon/src/project.ts imported profiles/native-lanes.ts, which profile:rows generates from packages/parity/out/lanes.json. That put the lanes verdict in the regen key of 21 steps. Whenever lanes-host rewrote lanes.json at the end of pass 1, for example when vectors changed and device lanes went stale, profile-rows rewrote native-lanes.ts in pass 2 and every compiling step reran. That pass took about 15 minutes and usually changed 0 files, and the same happened again after every device run. The PM approved this design (REGEN-SPEED decision 1).

What changed

  • Compiler. project.ts no longer imports native-lanes.ts. The verdict is a new option, InternalOptions.nativeLanes. Without it, a checked native output is analysis-only with the reason "no committed lanes verdict was given to this compilation". It is never ready, so a missing verdict never reads as passing. nativeOutputState takes null for that case. The native digest does not include the verdict, before or after.
  • Public API, unchanged. createProject moves to the new create-project.ts, which passes the committed NATIVE_LANES at the edge.
    • index.ts now exports api.ts (the shared public API, moved verbatim from index.ts) plus createProject. Its export list is the same.
    • internal.ts exports api.ts, so the internal entry, and every step that runs under --conditions=dragon-internal, no longer reaches the verdict.
  • Regen keying (scripts/regen-inputs.ts, scripts/regen.ts). Before, a workspace package was keyed on every export target under every condition, so 'dragon' always pulled in index.ts. Now:
    • commandOf records each node invocation's --conditions= flags. It refuses the -C and --conditions <x> spellings, which it would otherwise misread.
    • importClosure resolves a workspace package to the entries Node picks under each process's conditions, using a new exportTargetsUnder.
    • Called without conditions it behaves as before.
    • The trace check still fails any run whose process loads an entry the key left out, so the narrower key cannot miss an input. lanes-host's child processes, for instance, are checked the same way.
  • Result: native-lanes.ts is now an input of north-star only, which uses the public createProject. Before, it was an input of 21 steps.

Tests (each fails without the change it covers)

  • outputs-ready.test.ts, new test:
    • no verdict gives analysis-only, saying so;
    • a passing verdict gives ready on both targets, so the absent verdict is not read as a pass;
    • the public createProject gives exactly the committed-verdict outputs;
    • the digest does not depend on the verdict.
  • regen.test.ts, new test: closure under conditions. Internal-only, public-only, two processes, no conditions (unchanged), and Node's first-matching-key rule in exportTargetsUnder. The commandOf test now also checks the recorded conditions and the refused -C spelling.
  • Retargeted: regen.test.ts "changes the key for an import...". Its step now runs with --conditions=dragon-internal, so the internal-only file it edits is still an input, and the intent (an import changes the key) is kept. It now also checks that without the condition that file is no input.
  • Retargeted imports: dist.test.ts, media-sweep.test.ts and north-star-tree.test.ts now import createProject from dragon/src/index.ts. Their 'dragon' import resolves to the internal entry, which no longer exports it. Nothing else changes.

What passed

  • pnpm typecheck: pass.
  • vitest run: regen.test.ts (32 passed, 2 skipped opt-in), outputs-ready, ua, s5, media-wiring, android-target, grid and compile tests (157 passed).
  • pnpm regen on the branch (job regen-speed-lanes-edge-regen): 19 steps ran. Fixed point after 1 pass, 0 files changed, git status clean. Every output is byte-identical, and the trace guard accepted every narrowed key.

Note

project.ts is also edited by many in-flight PRs. This one touches only its import lines, the InternalOptions and Resolved types, one line of the compile loop, nativeOutputState, and the removed createProject.

No tolerance or check is loosened.

🤖 Generated with Claude Code

…ules, so ua:capture's regen inputs leave out the compiler
….ts; the compiler and the internal entry no longer import it, and regen keys a workspace package on the entry Node picks under the step's conditions
…ault)

planted-swift.test.ts and planted-kotlin.test.ts compiled and ran a planted harness for each of the 7 FAULTS in one test
(planted-swift took 734 s on the CI runner, planted-kotlin 329 s). Each (target, fault) pair is now its own test file
(planted.ts holds the shared test), so vitest schedules them across workers and shards. planted-union.test.ts proves the
files cover FAULTS on both targets exactly once.
The determinism describe of parity.test.ts (356 tests, 290 s of its 548 s on the CI runner) runs from
parity-determinism-<k>.test.ts, one chunk of FIXTURES per file under the same describe titles, each with its own Chrome and
its own coverage check. parity.test.ts keeps a coverage test proving the chunk files together check every fixture once.
… with any NaN; corpus inputs, results and digests write every NaN as 7ff8000000000000 (x86-64 makes some fff8...)
… are not serialized through Playwright's value serializer
@thejackshelton thejackshelton added the landing-failed The landing driver stopped this PR; see its comment label Oct 5, 2026
@thejackshelton

Copy link
Copy Markdown
Contributor Author

Landing stopped at step merge (pnpm land)

merge-train: merging native-lanes-at-edge onto 918aa25b016852bd6a44f68748c1ed9d5dbca5b8 failed; conflicts in packages/parity/test/regen.test.ts, scripts/regen.ts

Fix the cause, then hand #153 back to the landing queue with the new clean head.

@thejackshelton thejackshelton removed the landing-failed The landing driver stopped this PR; see its comment label Oct 5, 2026
thejackshelton and others added 9 commits October 5, 2026 00:13
The quiet gate required zero other heavy-slot holders and load under 20. A near-idle holder (a profiling run) kept the gate
shut at load ~1 while a batch's single failing file waited to be rerun. Below load 6 the machine counts as quiet regardless
of holders; the quiet request still stops new jobs from starting. land.test pins both sides.
digest: SHA-256 through Node's native hash when available (8x faster compiles, same digests)
regen: ua:capture's inputs leave out the compiler (platform.ts imports the reference platforms from their own modules)
SLOW-SPLIT 1: planted translator fault tests, one file per (target, fault)
…publish, never returned without it (the landing driver's ignored-file cleanup leaves such entries)
…utputs

packages/translate/out/{kotlin,swift}/ are content-keyed caches valid for any tree. #155's clean removed their files but not
their directories, which left stale entries that blocked every later build of the key (fixed at the source in #164); keeping
them also saves the harness rebuilds on every proof. land.test adds both to the kept set.
…try root cause), where each target stopped
thejackshelton and others added 24 commits October 5, 2026 01:08
translate: a stale harness cache entry (artifact deleted, directory kept) is replaced, never returned; root cause of the land-147 'Unable to access jarfile' failures
land: keep the translate harness build caches when cleaning ignored outputs
land: an idle machine is quiet even with a heavy slot held
SLOW-SPLIT 2: parity determinism shards in 4 chunk files
NaN-any: a NaN matches any NaN in the bit-exact corpus comparison (PM ruling)
tw-sweep: compile on every core; Chrome checks on 8 pages, each capture one JSON string (same snapshot)
Chrome captures: pixel, DPR and break captures take 8 cases at once (same bytes)
@thejackshelton
thejackshelton merged commit c5fde70 into master Oct 5, 2026
5 checks passed
@thejackshelton
thejackshelton deleted the native-lanes-at-edge branch October 5, 2026 06:59
thejackshelton added a commit that referenced this pull request Oct 5, 2026
…ers): the resize-capture step moves to scripts/regen-steps/mq-r1.ts (after capture, so before profile-rows), the media-runtime group to one sorted GROUPS line, .gitattributes sorted; hit suite 2092 beside master's animator suite
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant