Repository navigation
ci: bound the type-check Turbo cache save so bookkeeping cannot void a recorded verdict - #7047
Conversation
…a verdict
The `Type Check` job's verdict is recorded by its checking steps; everything
after them is bookkeeping. On 2026-08-26 that bookkeeping ate a verdict: all 21
real steps of the job succeeded and then `Post Turbo Cache` — the save phase of
`actions/cache@v6` — spent 13m09s in the upload and was still there when the
job's `timeout-minutes: 20` fired at 20m02s. The job went `cancelled`, the merge
queue cannot tell `cancelled` from `failure`, and an all-green pull request was
dequeued. Transient, not structural: the same job on the same head thirteen
minutes earlier saved that cache in one second.
Combined `actions/cache` declares `main: dist/restore/index.js` plus
`post: dist/save/index.js`, so the save is a step the runner generates and no
workflow key can bound it. Split it, per the action's own `save-always`
deprecation text:
- `actions/cache/restore@v6` — same position, path, key, restore-keys.
- `actions/cache/save@v6` — last, where the post phase already ran, with
`timeout-minutes: 5` and `continue-on-error: true`, skipping on an exact
primary-key hit exactly as the combined action did internally.
The ordering is written into the workflow comment, because it is what a future
reader needs in order to judge whether the step may be touched. The job ceiling
is deliberately unchanged: a larger ceiling buys a longer hang and still ends in
`cancelled`.
Every command-bearing step in every job of `ci.yml` stays byte-identical to
`main` and no job's `timeout-minutes` moves, so what this gate accepts and
rejects is untouched. `turbo-cache-save-bound.test.ts` pins the contract —
reverting the split is otherwise invisible: the cache still works, every run
still passes, and the next stall ejects the next green pull request.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GgDDqh6YnkXqsnVTCa7wHk
PM 验收 · R39 —— ACCEPT
围栏核对
⭐ 最值得记的一条是「直觉写法不成立」这个发现。 我给的方向是「给 save 设界」,而最自然的写法 —— 在现有 ⭐ 界值同样是测出来的:健康 save 1 秒,卡死那次 789 秒 ⇒ ⭐ 那条自己作废掉的消融,我认为是本轮全 lane 最有价值的方法记录:锚点用 裁定 1 —— 推广问题:选 B,已立卡 #7048你的建议 B 是对的,⛔ C 出局(charter 限定在 type-check job,横跨五个 workflow 扩大验证面不该骑在这张 PR 上)。已按 B 立卡。
⇒ ⭐ 你的报告是对的,是我先按较窄的那份材料行动了。#7048 已更正并记下了原因。 裁定 2 —— ceiling pin(
|
Two of the seven combined `actions/cache@v6` uses objectui#7048 measured live in this file, split per PR #7047's route: `actions/cache/restore@v6` in place and unbounded, `actions/cache/save@v6` last, bounded and `continue-on-error`. The save phase of the combined action is a step the runner generates at job end and no workflow key reaches it, so an upload stall runs the job into its ceiling and reports `cancelled` — which the merge queue cannot tell from `failure`. Each bound is derived from that site's own measurement, never inherited: - `e2e` / Playwright browsers (~269 MB), ceiling 30 min -> 8 minutes. A save is only observable after a version bump, so the bound is sized off measured restores of the same archive (3s here on jobs 100096569775 and 100094597323, 5s in live-e2e.yml job 100097466897) with a deliberately pessimistic 20x upload penalty: ~1m40s honest save, bound 4.8x that, demanding 0.56 MB/s sustained. 8 x 2 = 16 <= 30. - `docs` / `.turbo/cache`, ceiling 15 min -> 5 minutes. This job's own save measured 5s on job 100094597253 (a merge_group build, where these steps actually run). 5 min is 60x that; 5 x 2 = 10 <= 15. No job's `timeout-minutes` moves and no command-bearing step changes: all 7 jobs and 50 command steps of this file compare byte-equal against adbda1b, with a control proving the comparator non-vacuous. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b
Same split as ci.yml's: restore in place and unbounded, save last with `timeout-minutes: 5` and `continue-on-error: true`. Bound derived here, not inherited from PR #7047 — landing on the same number is a result: this job's own save of `.turbo/cache` measured 6s on 2026-09-02 (`Post Turbo Cache`, job 100097466846) against a 5m14s job, so 5 minutes is 50x the measured save. `Lint` is a required context on `pull_request` and `merge_group`, and this job declares NO `timeout-minutes` — its ceiling is GitHub's 360-minute default, so an unbounded stalled save could hold the shared serial queue for six hours before reporting `cancelled`. Adding a ceiling is deliberately NOT done here: it would change what this gate rejects. The bound is on the bookkeeping step alone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b
Same split; `timeout-minutes: 5`, `continue-on-error: true`, save placed last — after the budget verdict and after the comment that reports it, which is where the generated post phase already ran. Bound derived from this site: this job's own save of `.turbo/cache` measured 1s on 2026-09-02 (`Post Turbo Cache`, job 100096569559) against a 4m07s job, so 5 minutes is 300x it. That is the same RATIO PR #7047 sized its bound at, arrived at from this job's own number rather than copied from that one. `Bundle Analysis` is a required context (objectui#6245) and this job declares no `timeout-minutes`, so its ceiling is the 360-minute default — the bound is the only thing that stops a stalled save holding a required check open for hours. Adding a ceiling would change what the gate rejects and is deliberately not done. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b
Reverting any of the seven splits is invisible — the cache still works, every run still passes, and the next transient stall ejects the next green pull request. PR #7047 shipped a pin for exactly that reason; this one generalises it from the one incident site to the population, because the population is what this card found: seven more combined uses across five workflows, the repository's default spelling for caching. Enumerates every workflow file and requires of every `actions/cache/save`: `timeout-minutes` present, `continue-on-error: true`, placed after its job's last command step, same path and key as a matching restore with an `id`, and guarded on that restore's `cache-hit`. Where a job declares a ceiling, the SUM of its save bounds x 2 must fit inside it — the generalisation of #7047's invariant to a job with more than one cache. Non-vacuity is asserted on the axis that actually failed when this card was filed: the POPULATION, not the probe. A control on a probe cannot catch a scope inherited from someone else's narrative, which is how the first count came in 3.5x low. Deliberately not asserted: that restore halves are bounded (a restore stall fails before any verdict exists, which is honest), and any specific bound value — each site's number is derived from that site's own measurement and lives in the comment beside it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b
…a verdict (objectstack-ai#7269) * ci: bound the e2e and docs cache saves in ci.yml (objectstack-ai#7048) Two of the seven combined `actions/cache@v6` uses objectui#7048 measured live in this file, split per PR objectstack-ai#7047's route: `actions/cache/restore@v6` in place and unbounded, `actions/cache/save@v6` last, bounded and `continue-on-error`. The save phase of the combined action is a step the runner generates at job end and no workflow key reaches it, so an upload stall runs the job into its ceiling and reports `cancelled` — which the merge queue cannot tell from `failure`. Each bound is derived from that site's own measurement, never inherited: - `e2e` / Playwright browsers (~269 MB), ceiling 30 min -> 8 minutes. A save is only observable after a version bump, so the bound is sized off measured restores of the same archive (3s here on jobs 100096569775 and 100094597323, 5s in live-e2e.yml job 100097466897) with a deliberately pessimistic 20x upload penalty: ~1m40s honest save, bound 4.8x that, demanding 0.56 MB/s sustained. 8 x 2 = 16 <= 30. - `docs` / `.turbo/cache`, ceiling 15 min -> 5 minutes. This job's own save measured 5s on job 100094597253 (a merge_group build, where these steps actually run). 5 min is 60x that; 5 x 2 = 10 <= 15. No job's `timeout-minutes` moves and no command-bearing step changes: all 7 jobs and 50 command steps of this file compare byte-equal against adbda1b, with a control proving the comparator non-vacuous. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b * ci: bound the lint.yml Turbo cache save (objectstack-ai#7048) Same split as ci.yml's: restore in place and unbounded, save last with `timeout-minutes: 5` and `continue-on-error: true`. Bound derived here, not inherited from PR objectstack-ai#7047 — landing on the same number is a result: this job's own save of `.turbo/cache` measured 6s on 2026-09-02 (`Post Turbo Cache`, job 100097466846) against a 5m14s job, so 5 minutes is 50x the measured save. `Lint` is a required context on `pull_request` and `merge_group`, and this job declares NO `timeout-minutes` — its ceiling is GitHub's 360-minute default, so an unbounded stalled save could hold the shared serial queue for six hours before reporting `cancelled`. Adding a ceiling is deliberately NOT done here: it would change what this gate rejects. The bound is on the bookkeeping step alone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b * ci: bound both live-e2e.yml cache saves (objectstack-ai#7048) This job has two combined `actions/cache@v6` uses; both are split. The saves go last, ordered Playwright-then-fixture so the observed post-phase order is preserved (post steps run in reverse registration order — job 100097466897). Bounds derived per cache: - Playwright browsers (~269 MB) -> 8 minutes, from measured restores of the same archive (5s here, 3s in ci.yml) plus a pessimistic 20x upload penalty. - live backend fixture -> 5 minutes. This site's save measured 8s on 2026-09-02 (job 100097466897) — a real save, not a skip: the restore missed, which is why `Start ObjectStack backend` did the full clone and install. Both bounds share one 40-minute ceiling, so what matters is their sum: 5 + 8 = 13 minutes against a verdict path measured at 2m25s end to end, so even both bookkeeping halves stalling at once cannot reach the ceiling. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b * ci: bound the performance-budget.yml Turbo cache save (objectstack-ai#7048) Same split; `timeout-minutes: 5`, `continue-on-error: true`, save placed last — after the budget verdict and after the comment that reports it, which is where the generated post phase already ran. Bound derived from this site: this job's own save of `.turbo/cache` measured 1s on 2026-09-02 (`Post Turbo Cache`, job 100096569559) against a 4m07s job, so 5 minutes is 300x it. That is the same RATIO PR objectstack-ai#7047 sized its bound at, arrived at from this job's own number rather than copied from that one. `Bundle Analysis` is a required context (objectui#6245) and this job declares no `timeout-minutes`, so its ceiling is the 360-minute default — the bound is the only thing that stops a stalled save holding a required check open for hours. Adding a ceiling would change what the gate rejects and is deliberately not done. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b * ci: bound the changeset-release.yml Turbo cache save (objectstack-ai#7048) The release lane, treated with extra care: the diff is strictly the cache step and its comment, the save is appended LAST — after publish, after the post-version tree validation and after the refresh — and no step above it is touched. Publishing cannot be affected by a step that runs after all of it and carries `continue-on-error: true`. The derivation here is weaker than at the other six sites and the comment says so rather than dressing it up. A same-SITE save was not observable in the sampled runs: this job runs on `push` to `main` only when the version is not yet on npm, and on the 6-hourly `schedule` the primary key is an exact hit — the sampled scheduled run's `Post Turbo Cache` measured 0s with its restore at 4s (job 99021319583, 2026-08-29). The bound is therefore derived from the SAME CACHE at four sibling sites, whose saves measured 1s, 2s, 5s and 6s on 2026-09-02 (jobs 100096569559, 100096569872, 100094597253, 100097466846); 5 minutes is 50x the slowest of those. This job declares no `timeout-minutes` either, so its ceiling is the 360-minute default. Adding one is NOT attempted: the release act must never be truncated by a clock. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b * test(ci): pin every workflow cache save as bounded and non-fatal (objectstack-ai#7048) Reverting any of the seven splits is invisible — the cache still works, every run still passes, and the next transient stall ejects the next green pull request. PR objectstack-ai#7047 shipped a pin for exactly that reason; this one generalises it from the one incident site to the population, because the population is what this card found: seven more combined uses across five workflows, the repository's default spelling for caching. Enumerates every workflow file and requires of every `actions/cache/save`: `timeout-minutes` present, `continue-on-error: true`, placed after its job's last command step, same path and key as a matching restore with an `id`, and guarded on that restore's `cache-hit`. Where a job declares a ceiling, the SUM of its save bounds x 2 must fit inside it — the generalisation of objectstack-ai#7047's invariant to a job with more than one cache. Non-vacuity is asserted on the axis that actually failed when this card was filed: the POPULATION, not the probe. A control on a probe cannot catch a scope inherited from someone else's narrative, which is how the first count came in 3.5x low. Deliberately not asserted: that restore halves are bounded (a restore stall fails before any verdict exists, which is honest), and any specific bound value — each site's number is derived from that site's own measurement and lives in the comment beside it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGMDbrVa8JjZcCQ7DWYH1b --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #6577
The defect
On 2026-08-26 a pull request whose every check was green was ejected from the merge queue with
CI_FAILURE. Nothing in its diff failed. All 21 real steps of itsType Checkjob succeeded —Run type-checkreported success at 13:31:31Z — and thenPost Turbo Cache, the save phase ofactions/cache@v6, spent 13m09s inside the upload and was still there when the job'stimeout-minutes: 20fired at 20m02s. The job wentcancelled, the merge queue cannot tellcancelledfromfailure, and a pull request that had passed was dequeued.The verdict had already been recorded. Everything after 13:31:31Z was bookkeeping. A transient upload stall discarded an answer that was already in hand — and the bill lands on every lane, because the queue is one shared serial resource: 20 minutes of head-of-line blocking, an ejection, then a full re-run.
Transient, not structural. I re-measured both jobs from the Actions API rather than inheriting the card's numbers:
Turbo Cache(restore)Run type-checkPost Turbo Cache(save)A healthy save of this cache is one second. That measurement is what the bound below is sized against.
Mechanism, and why this one
The charter's stated direction is this repo's own objectui#5304
apt-getprecedent, already written intoci.yml: an unbounded network call whose only backstop was the job ceiling, which converts a transient fault into a CANCELLED check — a gate that reports nothing at all. Bound the call so the job survives it.The obvious spelling — put
timeout-minuteson the existingTurbo Cachestep — does not work, and I checked rather than assumed.actions/cache@v6's ownaction.ymldeclares:The save is a step the runner generates at job end (
Post Turbo Cache, step #40 in the incident), not the authored step. No workflow syntax attachestimeout-minutesorcontinue-on-errorto a generated post step. So the step is split, which is the routeactions/cache's ownsave-alwaysdeprecation text points at:actions/cache/restore@v6— same position, samepath,keyandrestore-keys.actions/cache/save@v6— runs last, which is exactly where the post phase already ran, carryingtimeout-minutes: 5andcontinue-on-error: true.Both sub-actions declare only
main:(nopost:), so both keys are ordinary documented step keys there.timeout-minutes: 5— 300x the measured healthy save, so it cannot fire on a working runner; and measured from the slowest verdict path on record (6m53s, the incident run) it lands ~8 minutes clear of the 20-minute ceiling, so this step can no longer reach it.continue-on-error: true— without it the bound would only trade acancelledgate for a red one. A cache that failed to upload costs the next run some time; it says nothing about the code under test, so it must not speak for it.Rejected alternatives
timeout-minutes— ruled out on the card and by triage. A larger ceiling buys a longer hang and still ends incancelled; lifting a gate's ceiling is gate-weakening. The job ceiling is unchanged at 20, and a pin now guards it.timeout-minuteson the combined step — cannot reach the post phase, per theaction.ymlabove.merge_group— bounds nothing (a restore stall would still hang), and it removes cache warming from the very lane that is most expensive to re-run.The ordering, written down
Per the dispatch, the sentence a future reader actually needs is now in
ci.ymlitself, at the restore step: this job's verdict is recorded by its checking steps; everything after them is bookkeeping, and bookkeeping must never be able to discard an answer the gate has already produced. That is the one thing needed to judge whether this step may be touched.The gate's verdict path is unchanged — evidence, not assertion
A step decides this job's verdict iff it runs a command and its failure is not swallowed. I compared that set between
origin/mainand this branch, for every job inci.yml, by parsing both revisions:Every command-bearing step in every job is byte-identical, including its
if:and its owntimeout-minutes, and no job's ceiling moved. The control run proves the comparison is not vacuous — it detects a planted change toRun type-check.The only two behavioural deltas are on the cache steps themselves, and both preserve what the combined action did internally:
cache-hit != 'true'reproduces the combined action's "exact hit on the primary key, do not save" skip.success()still applies — matching the combined action'spost-if: success().The pin, and its teeth
Reverting this fix is invisible: fold the split back into one step and the cache still works, every run still passes, and the next transient stall ejects the next green pull request. That is this repo's recurring "looks like enforcement, isn't" class, and it is why objectui#5304 shipped with
ensure-chromium-ready.test.tsbeside it.scripts/__tests__/turbo-cache-save-bound.test.tsis the equivalent here (6 assertions).Teeth proven by ablation against the committed tree. Each leg: mutate, prove the mutation on disk by counting both the removed and the injected text, run, then restore with
git checkout HEAD --and prove the restore by blob hash plus an emptygit diff HEAD.actions/cache@v6continue-on-error: true, keep the boundtimeout-minutes: 5survivesRun type-checkHEAD blob of
ci.yml9402516686before every leg and9402516686after every leg.One honest note on method: the first attempt at the middle ablation was rejected by its own on-disk check as a no-op and its reading discarded — my anchor was the phrase
continue-on-error: true, which also occurs in the comment this PR adds, so the count could not reach 0. Re-run with the anchor pinned to the YAML key at its exact indent (^ continue-on-error: true$); the table above is the re-run.Verified
Run on the final commit
1f1338fe0, all through the shared verify lock:pnpm exec vitest run scripts/__tests__/— 92 files, 2593 tests passed. This is the surface that pinsci.yml: 27 of those files read it. Baseline onorigin/mainbefore the change was 91/2587, so the delta is exactly the 6 new assertions.pnpm exec tsc -p tsconfig.scripts.json --listFiles— clean, and the new test file is confirmed in the program (1 hit), so this is a measurement rather than a silent skip.pnpm exec eslint scripts/__tests__/turbo-cache-save-bound.test.ts— clean. It caught a realno-useless-escapeon the first pass, fixed before commit.pnpm check:control-bytes— OK, 5851 tracked text files; plus a manual sweep of the three changed files.node scripts/check-changeset-presence.mjsandcheck-changeset-no-major.mjs— both green.node scripts/check-governed-queue-guard.mjs --teston all three paths — NOT GOVERNED, ordinary route.The workflow was also parsed with the
yamlpackage and its step list inspected directly, rather than judged by eye.Scope
This card is n=1 and stays that way — one measured occurrence, on the
type-checkjob. Two other jobs inci.yml(e2e,docs) still use the combined action and carry the same structural exposure, but neither has been observed to hang and generalising was not in the charter; that is a question for the PM, not a rider on this PR.Generated by Claude Code