Repository navigation
Commit eede467
ci: Test Core's two timing gates exit-grade on
Fixes #22949
Clause-②: no (internal CI tooling; no published surface)
Refs #22966
Refs #22968
Implements ruling
[6119186511](#22949 (comment))
(letter C2 for both timing steps, plus an absolute floor; the
maintainer's word 「当然短期先尽快修复」): **a timing gate exit-grades only on the
run profile its baseline was measured on; everywhere else it reports.**
#22966 (the PR-time drift card) is the drift half of the same ruling and
remains open for triage. #22968 (the hourly card) remains open until an
uneventful green hourly run after this lands.
## What changes
| Step (`ci.yml`) | `schedule` | `merge_group`, `pull_request`, `push`,
`workflow_dispatch` |
|:--|:--|:--|
| `Test Core` → "Check the per-package suite-duration ceilings" |
unchanged: a package over its ceiling exits 1 with an `::error::`
annotation | the same table prints, each package over its ceiling is a
`::warning::` annotation, exit 0 |
| `Test Core (N/6)` → "Check this shard's timing drift" | unchanged:
DRIFT past 1.5x exits 1 (and withholds the shard attestation) | the same
verdict text prints, the DRIFT is one `::warning::` annotation, exit 0 |
- **Absolute floor (ceiling step only):** a package whose effective
ceiling is under 20 s (`CEILING_FLOOR_SECONDS`) is never red on any
event and prints `below floor`. A ceiling at the floor is graded; a
ruled raise that lifts a ceiling to the floor or above grades the
package again.
- **Unchanged:** the formula (window max x 1.25),
`RULED_CEILING_RAISES`, the Q2 B rule for new packages, the dataset and
the refresh lane. No edit to `scripts/test-shard-timings.json`,
`scripts/measure-test-shard-timings.mjs` or
`.github/workflows/shard-timings-refresh.yml`.
- **Refusals stay red on every event.** A malformed raise record or
ceiling table, or a missing `--event`, is not a timing reading and does
not vary with runner load, so it exits 1 everywhere. The raise record is
already checked at PR time by lint.yml's `--self-test` ("the live record
is well-formed"). This is my reading of the ruling's scope (it names
"every package over its ceiling or shard over its drift bound"); if the
seat reads it as "the step never exits 1 off schedule", the change is
small and local to the two render functions.
- **`workflow_dispatch` reports.** The ruling names `schedule` as the
one exit-graded event. The refresh lane reads `event=schedule` runs only
(`DEFAULT_RUN_EVENT` in `scripts/ci/select-shard-timings-run.mjs`, no
`--event` override in `shard-timings-refresh.yml`), and a dispatched run
writes the remote turbo cache (`TURBO_CACHE` `rw`) where the hourly run
only reads it, so it is not the baseline's profile.
## H2: how the off-schedule path is spelled
Both scripts take a **required** `--event`, and ci.yml passes `--event
"$GITHUB_EVENT_NAME"`. The rule (`TIMING_GRADED_EVENT = 'schedule'`,
`timingGateMode`, `timingModeNote`) is defined once, in
`scripts/partition-test-shards.mjs`, and imported by
`scripts/check-test-suite-ceilings.mjs` (which already imported
`escapeWorkflowCommandMessage` from there).
- Why not `continue-on-error` (the `ci.yml:1204` spelling): it leaves
the `::error::` annotations in place and does not produce the
`::warning::` the ruling asks for.
- Why not an opt-in `--report-only` set by a YAML expression: the policy
would then live in an expression no self-test reads, and a typo in it
fails silently in one direction or the other. With `--event`, the
decision table is self-tested in the script, and a missing or empty
event is **refused** (exit 1), never defaulted to either mode.
- Neither step gains an `if:` or a `continue-on-error`. The drift step's
comment block carries "NO `if:` and NO `continue-on-error` (the red is
the point)"; that line still holds as written and is extended, not
removed. The other ⛔ lines H1 named (`ci.yml:1036`-`1037` and `:1218`)
bind the attestation pair, which this PR does not touch.
- The drift verdict was measured first: in graded mode DRIFT prints to
stderr with **no** annotation (pinned by the existing self-test), so
`partition-test-shards.mjs --check-drift` needed the same flag for a
warning to exist at all.
## The floor: 20 s, and its reading
Read by the `domain:devx` seat off the table's distribution, as the
ruling assigns: on `1f312fdfa` (unchanged at this PR's base `246edf5b3`)
the 72 ceilings run 3.40, 3.69, 4.24, 5.18, 6.39, 9.10 and 16.23 s, then
22.71 s (`downstream-contract`) and up. 20 s sits in the widest gap of
the low band (16.23 to 22.71). Seven packages fall under it:
`knowledge-ragflow`, `service-cache`, `observability`, `sdui-parser`,
`knowledge-memory`, `embedder-openai`, `client-react`. The value, its
reading and the ruling URL are recorded once, in the gate header ("The
absolute floor").
## H5: packages the hourly run never grades
The hourly run grades only the packages it executes; a turbo cache
replay carries no window. I read the "How much of the run this table
saw" block of six hourly `schedule` runs (MCP `get_job_logs` on each
`Test Core` job; the 04:09Z set is the complement of that run's graded
rows against the 72-package dataset, since the tail read missed its
block):
| Run | Time (2026-10-12) | Replayed packages |
|:--|:--|--:|
| `38192273698` | 01:32Z | 12 |
| `38195222269` | 02:17Z | 5 |
| `38198819852` | 03:12Z | 27 |
| `38202645321` | 04:09Z | 14 |
| `38206660804` | 05:08Z | 15 |
| `38210849670` | 06:05Z | 11 |
- **Replayed on every one of the six runs: none.** The 02:17Z run
replayed only `formula`, `metadata-fs`, `observability`, `sdui-parser`
and `spec`.
- **Replayed on all three runs since the table went live (04:09Z,
05:08Z, 06:05Z): `core`, `metadata`, `plugin-hono-server`.** Each was
also replayed at 01:32Z and 03:12Z (5 of 6); the one run that executed
them (02:17Z) predates the ceiling table, so none of the three has been
graded yet. `formula`, `metadata-fs` and `spec` were replayed on 4-5 of
the 6.
- **Handling, as the ruling leaves it:** graded on any hourly run that
executes them, report-only everywhere else. No mechanism forces them to
execute (not in the ruling); I do not think one is needed yet, because
the replay set rotates run to run, and the gate header now states the
blind spot.
## Parity checkers
None was touched; none needed an edit. Run green on the head unchanged:
`check-self-test-workflow-commands`, `check-step-collectors`,
`check-workflow-status-functions`, `check-required-contexts`,
`check-aggregator-roster`, `check-ratchet-remedy-authority`,
`check-stall-guard-budget`, `check-stall-guard-headroom`,
`check-workflow-step-name-quoting`, `check-shard-attestation`. H3
("expect at least one to need a matching edit") did not hold.
## Evidence (head `ef3d0764b`)
**Ceiling CLI, one synthetic capture** (`cloud-connection` 200.33 s vs
163.89 s, `downstream-contract` 25.00 s vs 22.71 s, `knowledge-ragflow`
4.80 s vs 3.40 s, `client` 150.00 s vs 179.74 s, against the live
dataset):
```text
--event schedule EXIT=1
::error title=Suite-duration ceiling::@objectstack/cloud-connection: 200.33s, over its 163.89s ceiling. ...
::error title=Suite-duration ceiling::@objectstack/downstream-contract: 25.00s, over its 22.71s ceiling. ...
-- @objectstack/knowledge-ragflow: 4.80s against 3.40s (1.41x) -- below floor: a ceiling under 20s is never red
exit-graded: a `schedule` run is the profile the baseline was measured on.
--event merge_group EXIT=0
::warning title=Suite-duration ceiling::@objectstack/cloud-connection: 200.33s, over its 163.89s ceiling. ... Report only on a `merge_group` run: ...
::warning title=Suite-duration ceiling::@objectstack/downstream-contract: 25.00s, over its 22.71s ceiling. ... Report only on a `merge_group` run: ...
(the same stderr table, line for line, ending)
report only: a `merge_group` run is not the profile the baseline was measured on (`schedule`), so a red reading is a warning here and the step exits 0.
no --event EXIT=1 Error: the run's event is null. ... A missing event is refused rather than guessed.
```
**Drift CLI, one synthetic turbo summary** (`runtime` 916.1 s,
`service-analytics` 463.7 s, `client` 183.5 s against the live weights =
1.82x):
```text
--event schedule EXIT=1 stdout empty; stderr "shard-timing-drift: DRIFT -- Test Core (6/6), 1563.3s measured vs 860.2s predicted ... = 1.82x" + "exit-graded: ..."
--event pull_request EXIT=0 one "::warning title=Test Core shard timing drift::Test Core (6/6): ... past the red bound. ... Report only on a `pull_request` run: ..." + the same stderr text ending "report only: ..."
no --event EXIT=1 Error: the run's event is null. ...
```
**Self-tests:** `check-test-suite-ceilings --self-test` OK, 58 cases
across 9 batteries (two new: the floor, 8 cases; off-`schedule`
reporting, 8 cases; roster floor 7 to 9). `partition-test-shards
--self-test` OK (one new battery, 8 cases; roster floor 12 to 13).
**Ablations** (each through `scripts/ablation-replace.mjs`, anchor hit 1
to 0, restored to the HEAD blob, `git diff HEAD` empty after):
- floor removed (the comparison against `CEILING_FLOOR_SECONDS` in
`gradeCeilings` rewritten to compare against 0): ceiling self-test red,
`floor: a 3.40s ceiling run at 4.20s read over / OVER, exit 1`.
- `timingGateMode` always `'grade'`: partition self-test red (`C2:
merge_group, pull_request, push, workflow_dispatch exit-grade, but only
schedule may`) and ceiling self-test red (`floor: a 22.71s ceiling run
at 25s off schedule was not a warning alone: exit 1`).
**Gates:** `dispatch-gates --commands` on this diff derives 59 commands
(the same 59 as at dispatch); `--ran` reconciliation: 59 accounted, 55
exit 0, 4 NOT MEASURED (exit 3, PREREQUISITE NOT MET:
`check:dts-closure`, `check:dual-build-cjs-loads`,
`check:lean-entry-closure`, `check:sourcemap-no-sources-content` read
built `dist/`; this diff touches no package, so no build is owed and CI
runs them). `pnpm check:pm-dispatch-gates`: fast tier 1828 cases, exit
0. The 9 workflow-valued families (the ceiling step's own `--captures`
run among them) are CI's.
**Lint, narrowed:** eslint over the two edited scripts, 2 files, 0
errors, 0 warnings (`--format json`). Population: `eslint.config.mjs`
lints `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}`, so both are in it; ci.yml
is not eslint's. Invariance: the config enables no type-aware linting
(`--print-config` shows `parserOptions` without `project`), so this diff
cannot move a verdict on any untouched file.
## Acceptance notes
- Size: 502 added / 84 removed lines over 3 files; about half the
additions are the three new self-test batteries, most of the rest is
header and step-comment prose. The shape is the one H6 listed (two step
edits, one flag in two scripts, a header, self-test cases).
- `ci.yml`'s comment above "Download this run's shard test timings"
still says the ceiling check "is the one step here that DOES red the
job". It remains true on `schedule`; I left it as is because the file
surface is the two timing steps. Carrier: none.
---
_Generated by [Claude
Code](https://claude.ai/code/session_012yK1ddAdAqfzwigRHZq3r2)_
---------
Co-authored-by: Claude <noreply@anthropic.com>schedule alone (C2), with a 20s ceiling floor (#22973)1 parent 712361b commit eede467
3 files changed
Lines changed: 502 additions & 84 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1066 | 1066 | | |
1067 | 1067 | | |
1068 | 1068 | | |
| 1069 | + | |
| 1070 | + | |
| 1071 | + | |
| 1072 | + | |
| 1073 | + | |
| 1074 | + | |
| 1075 | + | |
| 1076 | + | |
| 1077 | + | |
1069 | 1078 | | |
1070 | 1079 | | |
1071 | 1080 | | |
1072 | 1081 | | |
1073 | 1082 | | |
1074 | 1083 | | |
1075 | 1084 | | |
1076 | | - | |
1077 | | - | |
1078 | | - | |
1079 | | - | |
| 1085 | + | |
| 1086 | + | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
1080 | 1090 | | |
1081 | 1091 | | |
1082 | 1092 | | |
| |||
1109 | 1119 | | |
1110 | 1120 | | |
1111 | 1121 | | |
1112 | | - | |
| 1122 | + | |
1113 | 1123 | | |
1114 | 1124 | | |
1115 | 1125 | | |
| |||
1486 | 1496 | | |
1487 | 1497 | | |
1488 | 1498 | | |
1489 | | - | |
1490 | | - | |
1491 | | - | |
1492 | | - | |
1493 | | - | |
1494 | | - | |
| 1499 | + | |
| 1500 | + | |
| 1501 | + | |
| 1502 | + | |
| 1503 | + | |
| 1504 | + | |
1495 | 1505 | | |
1496 | 1506 | | |
1497 | 1507 | | |
| |||
1505 | 1515 | | |
1506 | 1516 | | |
1507 | 1517 | | |
| 1518 | + | |
| 1519 | + | |
| 1520 | + | |
| 1521 | + | |
| 1522 | + | |
| 1523 | + | |
| 1524 | + | |
| 1525 | + | |
| 1526 | + | |
| 1527 | + | |
| 1528 | + | |
| 1529 | + | |
| 1530 | + | |
| 1531 | + | |
1508 | 1532 | | |
1509 | 1533 | | |
1510 | 1534 | | |
| |||
1515 | 1539 | | |
1516 | 1540 | | |
1517 | 1541 | | |
1518 | | - | |
| 1542 | + | |
1519 | 1543 | | |
1520 | 1544 | | |
1521 | 1545 | | |
| |||
0 commit comments