Skip to content

Commit ef3d076

Browse files
committed
ci(ceilings): exit-grade the suite-duration ceilings on schedule alone, with a 20s floor
The ceiling check takes the run's event (`--event`, required). On `schedule`, the profile the ceilings were measured on, a package over its ceiling reds the step as before; on every other event the same table prints, each package over its ceiling is a `::warning::` annotation, and the step exits 0. Refusals stay red on every event. A package whose effective ceiling is under CEILING_FLOOR_SECONDS (20s) is never red and prints "below floor"; the header records the floor's reading. ci.yml passes `--event "$GITHUB_EVENT_NAME"` to both timing steps, which keep no `if:` and no `continue-on-error`. Claude-Session: https://claude.ai/code/session_012yK1ddAdAqfzwigRHZq3r2 Co-authored-by: Claude <noreply@anthropic.com>
1 parent e6c7a6c commit ef3d076

2 files changed

Lines changed: 314 additions & 54 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 36 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1066,17 +1066,27 @@ jobs:
10661066
# second red on wall time against `timeout-minutes`. Both constants, their
10671067
# measured basis and a self-test battery each live in the script.
10681068
#
1069+
# THE RED IS A `schedule` RED ALONE (C2, both timing steps, ruling
1070+
# https://github.com/objectstack-ai/objectstack/issues/22949#issuecomment-6119186511):
1071+
# the dataset's weights are measured on the hourly `schedule` runs, so
1072+
# only those exit 1 on DRIFT. On `merge_group`, `pull_request`, `push`
1073+
# and `workflow_dispatch` the step prints the same verdict, raises the
1074+
# DRIFT as a `::warning::` annotation and exits 0. The event reaches the
1075+
# script as `--event "$GITHUB_EVENT_NAME"`, which it requires; the rule
1076+
# and its self-test live there (TIMING_GRADED_EVENT, timingGateMode).
1077+
#
10691078
# EXECUTED WINDOWS ONLY. Measurements come from samplesFromSummary() in
10701079
# measure-test-shard-timings.mjs, which drops cache HITs and failed
10711080
# suites, and the prediction is summed over the same executed packages.
10721081
# A replay's near-zero window would read as a shard far FASTER than
10731082
# predicted and vouch for a rotted dataset, so a shard whose packages all
10741083
# replayed is NOT MEASURED, never "fast".
10751084
#
1076-
# NO `if:` and NO `continue-on-error` (the red is the point), and ABOVE
1077-
# the attestation pair for the #6082 reason documented on the upload
1078-
# above: a failed unguarded step skips that pair, so a drift red also
1079-
# withholds this shard's attestation, the fail-closed direction. A shard
1085+
# NO `if:` and NO `continue-on-error` (the red is the point, and the
1086+
# `schedule`-only red is the script's `--event`, so neither is needed),
1087+
# and ABOVE the attestation pair for the #6082 reason documented on the
1088+
# upload above: a failed unguarded step skips that pair, so a drift red
1089+
# also withholds this shard's attestation, the fail-closed direction. A shard
10801090
# with no packages runs no turbo and writes no summary; that is NOT
10811091
# MEASURED here rather than a usage error from the script.
10821092
#
@@ -1109,7 +1119,7 @@ jobs:
11091119
exit 0
11101120
fi
11111121
node scripts/partition-test-shards.mjs --check-drift "${SUMMARIES[@]}" \
1112-
--label "Test Core (${{ matrix.shard }}/6)"
1122+
--label "Test Core (${{ matrix.shard }}/6)" --event "$GITHUB_EVENT_NAME"
11131123
11141124
# Runs even when the suite failed — that is when it earns its keep. It
11151125
# answers TWO questions about a red suite, and needs both to be able to
@@ -1486,12 +1496,12 @@ jobs:
14861496
# ── The per-package suite-duration ratchet (#16468) ──────────────────
14871497
#
14881498
# A GATE, unlike the two reporting steps above: NO `continue-on-error`,
1489-
# so a package whose suite ran past its ceiling reds `Test Core`, naming
1490-
# the package and its slowest test files. The ceiling is the package's
1491-
# slowest executed run in the refresh window x 1.25, generated into
1492-
# scripts/test-shard-timings.json (`ceilings`) by the refresh lane and
1493-
# held by every later refresh; it rises only by a ruled entry in the
1494-
# script's RULED_CEILING_RAISES. A `provisional` package prints "no
1499+
# so on a `schedule` run a package whose suite ran past its ceiling reds
1500+
# `Test Core`, naming the package and its slowest test files. The
1501+
# ceiling is the package's slowest executed run in the refresh window
1502+
# x 1.25, generated into scripts/test-shard-timings.json (`ceilings`) by
1503+
# the refresh lane and held by every later refresh; it rises only by a
1504+
# ruled entry in the script's RULED_CEILING_RAISES. A `provisional` package prints "no
14951505
# ceiling: provisional" and is not red; a package absent from the dataset
14961506
# prints "no ceiling: new package" and is not red until the first refresh
14971507
# after it lands gives it a ceiling, while a package the dataset knows
@@ -1505,6 +1515,20 @@ jobs:
15051515
# samplesFromSummary, and every vitest file line); a capture that did not
15061516
# arrive leaves its packages ungraded and says so, never red.
15071517
#
1518+
# EXIT-GRADED ON `schedule` ALONE (C2, both timing steps, ruling
1519+
# https://github.com/objectstack-ai/objectstack/issues/22949#issuecomment-6119186511):
1520+
# the ceilings come from the hourly `schedule` runs, so only those red
1521+
# this job on a package over its ceiling. On `merge_group`,
1522+
# `pull_request`, `push` and `workflow_dispatch` the step prints the same
1523+
# table, raises each package over its ceiling as a `::warning::`
1524+
# annotation and exits 0; a real slowdown merged to `main` reds the next
1525+
# hourly run. A package whose ceiling is under the absolute floor (20s,
1526+
# CEILING_FLOOR_SECONDS) prints "below floor" and is never red. The
1527+
# event reaches the script as `--event "$GITHUB_EVENT_NAME"`, which it
1528+
# requires; a refusal (a malformed table or raise record) is red on every
1529+
# event. The rule, the floor's reading and their self-tests live in the
1530+
# script.
1531+
#
15081532
# NOT MEASURED, exit 0, while the dataset carries no `ceilings`: they are
15091533
# generated only where the run-summary artifacts are downloadable (the
15101534
# refresh lane), so the first refresh after this lands writes them, and
@@ -1515,7 +1539,7 @@ jobs:
15151539
# captured, so a red shard's packages are graded too.
15161540
- name: Check the per-package suite-duration ceilings
15171541
if: always()
1518-
run: node scripts/check-test-suite-ceilings.mjs --captures "$RUNNER_TEMP/test-timings" --label 'Test Core'
1542+
run: node scripts/check-test-suite-ceilings.mjs --captures "$RUNNER_TEMP/test-timings" --label 'Test Core' --event "$GITHUB_EVENT_NAME"
15191543

15201544

15211545
# ── Temporal conformance against live, non-UTC servers (ADR-0053 D-A3) ─────

0 commit comments

Comments
 (0)