Skip to content

test-files.yml: run chosen test files on their full-test runners, dispatched and read over REST by pnpm ci:test-files - #221

Merged
thejackshelton merged 24 commits into
masterfrom
ci-test-files
Oct 8, 2026
Merged

thejackshelton merged 24 commits into
masterfrom
ci-test-files

Conversation

@thejackshelton

@thejackshelton thejackshelton commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Item 3 of the cloud migration plan (docs/goals/milestone-2-proof/notes/cloud-migration.md): a CI path that runs chosen test files on the right runner, for cloud sessions that have neither a Mac nor the native toolchains.

What changed

  • .github/workflows/test-files.yml (workflow_dispatch: ref, files, floor_write, optional nonce). The run-name is test files of <ref>, followed by (<nonce>) when a nonce is given.
    • resolve job:
      • Sorts the files with this workflow's own scripts/test-shards.ts group-files, read against the requested tree, so a branch that predates the mode can still be tested.
      • Also runs the ref's own test-shards.ts all and fails if any requested file's group differs. full-test.yml at that ref would run the file in that other group.
      • An unknown file fails the run before any test starts.
      • Uploads test-files-request: request.json with the ref, the resolved sha, the nonce, and the files with their groups.
    • Group jobs: chrome (macos-26), native (ubuntu-24.04-arm, Swift 6.4.0 + kotlinc 2.4.20 + JDK 17) and platform-free (ubuntu-latest). Each one copies full-test.yml's workflow env, job env, runner, timeout, setup steps, vitest command and "No native run was blocked" scan. Only the source of the file list and the report's name differ, plus DRAGON_FLOOR_WRITE/DRAGON_PIN_WRITE, which are empty unless floor_write is set. full-test.yml has no composite action, and it is left untouched.
      • test-shards.test.ts enforces this. Each of these edits fails it: an extra job env var, a changed DRAGON_WPT_DIR, an extra workflow env var, a disabled blocked grep, if: false on the blocked step, --passWithNoTests -t smoke in the vitest command, -t in the vitest list command, or a dropped step in the native job.
    • "Every requested file ran": checked in each job and again in a summary job over all the reports. It fails if:
      • a requested file is missing from the vitest JSON or appears twice;
      • a file's passed-plus-failed count differs from what an unfiltered vitest list collects for it. vitest reports tests dropped by a -t filter as skipped, and vitest list leaves out statically skipped tests, so this catches filtering;
      • every test in a file was skipped;
      • the report holds a file nobody requested.
    • floor_write: tests run with DRAGON_FLOOR_WRITE=1 and DRAGON_PIN_WRITE=1. Each job uploads test-files-floor-<group>/floor.patch, covering the merge=dragon-floor files in .gitattributes.
    • Concurrency: per job, grouped by ref plus a hash of the files plus the group, with cancel-in-progress: false. Workflow-level concurrency can't compute a hash.
  • scripts/test-shards.ts group-files [--root <dir>] <file>...: prints file<TAB>group. It refuses anything that is not exactly a test file of the tree.
  • pnpm ci:test-files <ref> <file>... [--floor-write] [--from <branch>] [--once], and --run <id> [--once] (scripts/ci-test-files.ts).
    • Uses REST calls only: gh api for the dispatch (with return_run_details) and the artifact list, gh run view, gh run list, gh run download.
    • Each dispatch carries a random nonce. If the dispatch doesn't return a run id, the fallback matches the exact run-name with that nonce and the dispatching branch, with no time window.
    • Verdict: prints the workflow branch and sha, the ref and the sha it resolved to, and the files with their groups. In dispatch mode it refuses a run whose request has a different ref, nonce or file set (exit 2).
    • The script repeats the "every requested file ran in full" check on the downloaded reports and lists, and prints every failing test.
    • Exit codes: 0 passed, 1 failed, 2 usage error, gh error, cancelled run (nothing judged) or wrong run, 3 pending (--once, or past the wait limit).

No tolerance, check, test or fixture was changed or removed.

What passed (head 7e7f8fe)

Notes:

  • GitHub returns 404 for a dispatch by file name until the workflow path is registered. While test-files.yml is not on master, I registered the path once with a throwaway branch, since deleted. After landing it is registered for good.
  • GitHub files a run's checks under the head of the branch it was dispatched from. An earlier proof dispatched from this branch left failing test-files checks on the old head 7413904.

🤖 Generated with Claude Code

…Apps kept the host in the scaled window one scene rotation left, so capture trust's OS screenshot showed the wallpaper
# Conflicts:
#	packages/parity/test/device-run.test.ts
…t lanes rerun, so the device records await the landing driver's device run)
…h pnpm ci:test-files to dispatch and read it over REST
…ems 1 and 4)

scripts/cloud-setup.sh, run by the project SessionStart hook only when CLAUDE_CODE_REMOTE=true, installs the Node every
workflow pins and the pnpm of packageManager, then pnpm install --frozen-lockfile and pnpm setup:git, which now also turns
rerere off. DRAGON_REQUIRE_NATIVE=1 turns a native run with a missing toolchain from blocked (owner tooling) into a failure
naming the tool, in runTarget and runHostLane, as CI's "No native run was blocked" step does.
…atch mode)

regen-on-ci.yml takes a sha (patch mode): no push, run-name "regen of <sha>", the combined
outputs.patch uploaded as regen-patch, every cache restored only. land-devices-ci.ts adds the
regen workflow, land-regen/c-<sha> scratch branches and the patch application; land.ts sends its
four regen call sites and the parallel preparations through it.
…group; ci:test-files reads reports strictly and removes them
…then the merge drivers)

floor-merge.test.ts pins pnpm setup:git to exactly what the landing driver sets; rerere.enabled false joins both, so the
exact-equality check stays and covers the new line.
The master warm run no longer yields to patch-mode runs; a regen step cut off (timeout, lost
runner, cancel) is CiUnavailable, only one that completed with failure blames the tree; patch mode
does not use setup-node's saving pnpm cache; DEFAULT_REGEN_WAIT_S covers the 6-round chain; a
commitApart error is CiUnavailable; the regen commit names the CI run; abandonCiRun says CI run.
…_NATIVE=1, prove the Linux path in CI

Review of 892a687: the hook now has no matcher, so it reruns after compact, clear and fork (compaction drops CLAUDE_ENV_FILE
exports); in a cloud session the script writes export DRAGON_REQUIRE_NATIVE=1 to CLAUDE_ENV_FILE first (PM ruling) and fails
without that file; only the summary line reaches stdout; npm --prefix; curl --retry-connrefused. ci.yml's checks job runs on
ubuntu-24.04 and runs cloud-setup.sh twice before setup-node, checking Node, pnpm, the export, typecheck and no reinstall.
…re; a nonce finds the dispatched run; every test vitest collects must run; the setup test covers env, vitest command and blocked scan
Commands: pnpm regen; pnpm run parity:devices; pnpm regen
Commands: pnpm regen; pnpm run parity:devices; pnpm regen
thejackshelton and others added 5 commits October 8, 2026 01:55
iOS simulators boot pinned to Full Screen Apps (iPad capture-trust failures in queues 59 and 60)
Land: LAND_REGEN=ci runs the driver's regens on CI (regen-on-ci.yml patch mode)
Cloud session bootstrap and DRAGON_REQUIRE_NATIVE (cloud migration items 1 and 4)
@thejackshelton
thejackshelton merged commit 364cf16 into master Oct 8, 2026
5 checks passed
@thejackshelton
thejackshelton deleted the ci-test-files branch October 8, 2026 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant