Skip to content

Commit e0c768f

Browse files
ci(dogfood): carry the shard slice in a task env so the dogfood test hash sees its upstream closure (#21074)
Fixes #20820 Clause-②: no ## What The required `Dogfood Regression Gate (k/3)` leg ran `turbo run test --filter=@objectstack/dogfood --only -- --shard=k/3`. `--only` drops the `^build` closure out of the `dogfood#test` hash, so a PR that changed only an upstream package could match a main-seeded cache entry and be replayed green. This ports #19278's shape to the dogfood leg: - `turbo.json`: `@objectstack/dogfood#test` now declares `OS_TEST_SHARD` in its `env`. The shared `test` task already declares it (#20824), but a per-package task entry REPLACES the shared entry's `env` instead of extending it, so the shared declaration did not cover dogfood: measured, the pre-change plan listed only `OS_TEST_TIERS` for that task. - `packages/qa/dogfood/vitest.config.ts`: reads `process.env.OS_TEST_SHARD` into the root `test.shard` (same spelling as `packages/cli/vitest.config.ts`). - `.github/workflows/ci.yml`: the leg exports `OS_TEST_SHARD="k/3"` and runs `pnpm turbo run test --filter=@objectstack/dogfood --log-order=stream`: no `--`, no `--only`, no `--force`. Stall guard, `dogfood.log` tee, `NODE_OPTIONS` / `--report-dir`, cache restore/save are untouched. The step's comment states the new mechanism. Also touched (comment only, one deviation from the card's file surface, flagged): the comment of the preceding closure-build step said it exists "so that the sharded run in the next step can be `--only`", which is now false; it now says so. The step itself is unchanged. No second shard mechanism. `scripts/measure-test-shard-timings.mjs` / `partition-test-shards.mjs` / `report-test-timings.mjs` are untouched: the dogfood leg passes no `--summarize` and has no timings consumer, so there is nothing to reuse or change. ## Measurements turbo 2.10.10 dry-runs (`--dry=json`) and real vitest runs, in this worktree at head 55bd9e0. Pin 1, hash moves (shard 1/3, edit = append a comment line to the file, restored from HEAD after): | leg | clean | + `packages/objectql/src/index.ts` | |---|---|---| | old (`--only -- --shard=1/3`) | a963b53363c79e20 | a963b53363c79e20 (unchanged: the defect) | | new, pre-build tree | d6f169ed1431553e | 2ee13c0afd852c96 (moves) | | new, post-build tree | bd114335fad78510 | 55dc2dd1da35f8e0 (moves) | The pre-build and post-build clean hashes differ (d6f169ed vs bd114335); I did not isolate why (suspect built output under the `examples/app-showcase/**` input). Each row's before/after is on one tree. Pin 2, lit control (post-build tree, shard 1/3, clean bd114335fad78510): - `packages/client-react/src/index.tsx` (a workspace package NOT in the dogfood closure, checked against the plan): bd114335fad78510, unchanged. - `packages/qa/dogfood/test/action-params-contract.dogfood.test.ts` (dogfood-internal): a5dc81d7aadebe29, moves. Pin 3, shards differ: post-build 1/3 bd114335fad78510, 2/3 0387e6b5b568a93c, 3/3 d3b158ab11f62582; pre-build d6f169ed1431553e / fd1f57f4bbe9537f / bc48cac953e838cc. Pins 1 to 3 are dry-run hash readings, not executions. Pin 4, the shard is honoured: vitest 4.1.11 (lockfile). Real `OS_TEST_SHARD=k/3 vitest run` in `packages/qa/dogfood` ran 50 / 49 / 49 test files (148 in total, 148 in the union, 0 overlap between any two shards), 0 failed in each. `vitest list --filesOnly` is NOT a valid measure of this: it prints all 148 files for any shard, even with `--shard=1/3` on the command line. #21032's vitest 5.0.2 is NOT MEASURED; the env-read shape is the one `packages/cli/vitest.config.ts` already uses, so the dogfood read breaks together with it if a vitest line stops honouring `test.shard`. Plan cost of dropping `--only` (`OS_TEST_SHARD=1/3 turbo run test --filter=@objectstack/dogfood --dry=json`): 66 tasks. Against the cache the step above fills: 63 HIT / 3 MISS (the test, plus two `#build` nodes of packages with no `build` script, which never execute); cold cache: 66 MISS. The leg's 65 `build` hashes are identical to `turbo run build --filter=@objectstack/dogfood`'s plan (65 same, 0 differ, 0 missing), so the closure replays rather than rebuilds. The old leg planned 1 task. Pin 5, nothing is replayed green: a replay needs an equal `dogfood#test` hash. That hash now carries the shard value (pin 3), the upstream closure (pin 1) and the package's own files (pin 2), the control shows it is not moving on everything, and pin 4 shows the shard value is what selects the files that execute. NOT MEASURED: an end-to-end executed replay across an upstream edit on the new leg. ## Gates `node scripts/pm/dispatch-gates.mjs --commands`: 79 derived; 78 run, all exit 0, recorded with `--ran`. `pnpm check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT MET: `Run pnpm build first`, nothing measured), so it is NOT MEASURED; the diff changes no built output. `pnpm --filter @objectstack/dogfood typecheck` exit 0, but that program does not compile `vitest.config.ts` (0 hits in `tsc --listFiles`), so the `satisfies` clause is checked only by vitest loading the file. actionlint is not installed here: NOT MEASURED; `ci.yml` parses as YAML and `turbo.json` as JSON. ## Changeset None. `@objectstack/dogfood` is `"private": true`, and `ci.yml` / `turbo.json` ship in no package's `files[]`. ## Acceptance notes None beyond the above. --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01VDtqoecgES7ScQYGbFVDRv --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 94608a7 commit e0c768f

3 files changed

Lines changed: 53 additions & 25 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1632,8 +1632,8 @@ jobs:
16321632
restore-keys: |
16331633
${{ runner.os }}-pnpm-store-v3-
16341634
1635-
# Shard-scoped key: the turbo test hash differs per shard (pass-through
1636-
# args are part of the task hash). Restore-only on PRs — see the Restore
1635+
# Shard-scoped key: the turbo test hash differs per shard (the task-declared
1636+
# env `OS_TEST_SHARD` is part of the task hash). Restore-only on PRs — see the Restore
16371637
# Turbo cache comment in the test job; the save step at the end of this
16381638
# job seeds from main only. The job-level catch-all (same as Test Core's)
16391639
# is what keeps a NEW shard number warm before main has ever saved it:
@@ -1653,13 +1653,16 @@ jobs:
16531653
run: pnpm install --frozen-lockfile
16541654

16551655
# ⛔ THE DOGFOOD PACKAGE'S DEPENDENCY CLOSURE IS BUILT HERE, IN A RUN
1656-
# THAT CARRIES NO PASSTHROUGH, so that the sharded run in the next step
1657-
# can be `--only` (#16395's shape, applied to this job by #16886).
1658-
#
1659-
# Turbo folds a run-level passthrough into the hash of EVERY task in the
1660-
# run, not only the task that receives it -- and `-- --shard=k/3` is the
1661-
# whole reason each shard gets its own invocation at all (the next step's
1662-
# comment says what those args are for). Measured on turbo 2.10.10,
1656+
# THAT CARRIES NO PASSTHROUGH. (#20820: the next step no longer needs this
1657+
# for correctness -- it carries its shard in the task env `OS_TEST_SHARD`,
1658+
# so it plans and replays this same closure itself, with no `--only`. The
1659+
# step stays as its own stall-guard site, and the closure replays there.
1660+
# The measurement below is the history of the `-- --shard=k/3` shape.)
1661+
#
1662+
# (History.) Turbo folds a run-level passthrough into the hash of EVERY
1663+
# task in the run, not only the task that receives it -- and the old leg's
1664+
# `-- --shard=k/3` was the whole reason each shard got its own invocation
1665+
# (the next step's comment says what carries the slice now). Measured on turbo 2.10.10,
16631666
# `--filter=@objectstack/dogfood`, `turbo run test ... --dry=json`
16641667
# (67 tasks: 66 `build` + 1 `test`):
16651668
#
@@ -1704,11 +1707,16 @@ jobs:
17041707
# Boots real example apps in-process (in-memory SQLite) and exercises them
17051708
# through the real HTTP + service stack — catches runtime regressions that
17061709
# build / unit tests / spec-liveness pass over (e.g. the #2018 tz-bucketing
1707-
# break, which was green on every static gate). The `--` args reach the
1708-
# package's `vitest run` and are hashed into the turbo task, so each
1709-
# shard caches its test leg independently -- and under `--only` that is
1710-
# the ONLY task the passthrough is hashed into; the build closure it used
1711-
# to scatter is built once, shard-independently, by the step above.
1710+
# break, which was green on every static gate). The shard slice travels in
1711+
# the task-declared env `OS_TEST_SHARD` (#20820, #19278's carrier):
1712+
# `turbo.json` lists it in `@objectstack/dogfood#test`'s `env`, so it is
1713+
# hashed into THAT task only, and `packages/qa/dogfood/vitest.config.ts`
1714+
# hands it to vitest's `shard`. There is no `--` passthrough and no
1715+
# `--only`: the run plans the `^build` closure with the same hashes as the
1716+
# step above (they replay), and the test task's hash therefore moves when
1717+
# an upstream package changes -- the old `--only` + `-- --shard=k/3` shape
1718+
# dropped the closure from the test's hash and a main-seeded cache entry
1719+
# could replay a shard green across an upstream-only change.
17121720
# run-with-stall-guard: same wiring as Test Core (see the comment there;
17131721
# #4250/#4314) — it tees to dogfood.log itself for the completeness guard
17141722
# below, propagates the suite's real exit status (no `| tee` + pipefail),
@@ -1728,17 +1736,14 @@ jobs:
17281736
# globalPassThroughEnv entry or turbo strips it — see the script header.
17291737
export VITEST_MAX_WORKERS="$(node scripts/vitest-worker-cap.mjs)"
17301738
mkdir -p "$RUNNER_TEMP/stall-reports"
1731-
# `--only` (#16395's flag, #16886's site): the step above already
1732-
# built this package's dependency closure in a passthrough-free run,
1733-
# so this run must schedule the ONE task the passthrough is for.
1734-
# Without it turbo re-hashes the whole `^build` closure under
1735-
# `--shard=k/3` and rebuilds it, once per shard -- the step comment
1736-
# above carries the measurement. ⚠ That build step is load-bearing
1737-
# for this flag: with the closure unbuilt, this run fails LOUDLY
1738-
# here (imports resolve to a missing dist), never as a silent green.
1739+
# The slice is the ENV below, not a passthrough: see the step comment.
1740+
# The build step above still builds the closure once, so the closure's
1741+
# `build` tasks REPLAY here (identical hashes: nothing run-level is
1742+
# hashed into them any more) and only `dogfood#test` executes.
1743+
export OS_TEST_SHARD="${{ matrix.shard }}/3"
17391744
node scripts/run-with-stall-guard.mjs --log "$RUNNER_TEMP/dogfood.log" --stall-minutes 10 \
17401745
--report-dir "$RUNNER_TEMP/stall-reports" -- \
1741-
pnpm turbo run test --filter=@objectstack/dogfood --only --log-order=stream -- --shard=${{ matrix.shard }}/3
1746+
pnpm turbo run test --filter=@objectstack/dogfood --log-order=stream
17421747
17431748
# Dogfood boots real apps in-process, so a native/OOM abort is likelier
17441749
# here than in the unit suites — and a shard that dies silently looks like

‎packages/qa/dogfood/vitest.config.ts‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@
4040
// HERE, in the harness, where the test author can see it.
4141
import { defineConfig } from 'vitest/config';
4242
import path from 'path';
43-
import { parseCLI } from 'vitest/node';
43+
import { parseCLI, type TestUserConfig } from 'vitest/node';
4444
import {
4545
exactAndGlobPopulations,
4646
runFilterPreflight,
@@ -111,8 +111,31 @@ runProjectCliOverridePreflight({
111111
parse: parseCLI,
112112
});
113113

114+
// #20820 -- THE FILE-LEVEL SLICE ARRIVES AS AN ENV VAR, NOT AS A PASSTHROUGH.
115+
// The `Dogfood Regression Gate (k/3)` leg runs `OS_TEST_SHARD=k/3 turbo run test`,
116+
// the same carrier `packages/cli/vitest.config.ts` documents (#19278). The value
117+
// reaches vitest HERE because vitest reads no shard variable of its own, and it
118+
// reaches this process at all only because `turbo.json` declares `OS_TEST_SHARD`
119+
// in THIS package's `@objectstack/dogfood#test` task `env` (a per-package task
120+
// entry REPLACES the shared `test` entry's env, it does not extend it -- the
121+
// shared declaration does not cover this package) -- which is also what puts the
122+
// slice in the task hash. Unset (every local run) it is `undefined` and the run
123+
// is unsharded; a `--shard` on the command line still wins, because vitest
124+
// merges the CLI options OVER this block.
125+
//
126+
// Why a passthrough (`-- --shard=k/3`) is no longer the carrier: turbo folds a
127+
// run-level passthrough into the hash of every task in the run, so the leg had
128+
// to be `--only`, and `--only` drops the `^build` closure out of the test's hash
129+
// -- a shard could replay a main-seeded cache entry across an upstream change
130+
// that put it in the affected set. An env declared on the task reaches only the
131+
// task. `shard` is typed through `TestUserConfig` because vitest declares it on
132+
// its CLI options and not on `InlineConfig`; it sits on the ROOT `test` block
133+
// (the projects below do not carry it), where vitest resolves it for all of them.
114134
export default defineConfig({
115135
test: {
136+
// The file-level slice, when the dogfood gate runs one (#20820) -- see the
137+
// section above `export default` for why it is spread and typed this way.
138+
...({ shard: process.env.OS_TEST_SHARD } satisfies Pick<TestUserConfig, 'shard'>),
116139
projects: [
117140
{
118141
test: {

‎turbo.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -418,7 +418,7 @@
418418
"@objectstack/dogfood#test": {
419419
"dependsOn": ["^build"],
420420
"outputs": [],
421-
"env": ["OS_TEST_TIERS"],
421+
"env": ["OS_TEST_TIERS", "OS_TEST_SHARD"],
422422
"inputs": [
423423
"$TURBO_DEFAULT$",
424424
"!dist/**",

0 commit comments

Comments
 (0)