Repository navigation
Commit ee8f7c3
fix(ci): collect every self-test verdict in a step, instead of masking after the first failure (#10887)
* fix(ci): collect every self-test verdict in a step, instead of masking after the first failure
GitHub runs a `run:` block as `bash -e <file>`, so the first non-zero exit
aborts the step and every command after it is never reached -- neither green
nor red, and nothing in the log tells those two apart.
Measured, not hypothetical: lint.yml's `Shallow-history guard self-tests` ran
three independent self-tests as a bare sequence with
`git-history.mjs --self-test` first, and that one was red on `main` for about
ten hours on 2026-08-21 (#10807). For the whole of that window
`check-engine-split-ratio.mjs --self-test` and
`collect-release-notes.sh --self-test` did not execute in CI once, on the step
that gates every PR. Both were green, so the mask cost nothing that day; the
expensive shape is the one it leaves behind, where a second regression lands
unnoticed while the first is red and then reads as though the fix broke it.
Both steps that had this shape now run every self-test unconditionally through
a `run_self_test` collector, print a verdict per self-test, and exit non-zero
at the end naming every one that failed.
Not a split into one step per self-test: a plain split does not fix this at
all, because Actions skips a job's remaining steps once a step fails, so the
mask survives the split verbatim. Restoring the property would take an `if:`
on each gate step, and a condition is a way for a PR to arrange that a gate
does not run on it. Both gates that read step structure were checked and would
have tolerated a split -- `check-shard-attestation` scans ci.yml only, and
`check-required-contexts` pins job-level properties plus the single
`check:required-contexts` step -- so the split was rejected on the merits, not
because a gate refused it.
`scripts/check-step-collectors.mjs` closes the class in both halves. Statically
it requires any `run:` block invoking `--self-test` on two or more DISTINCT
scripts to route them through a collector, and deliberately does not flag the
`<gate> --self-test` + `<gate>` precondition shape that dominates this job.
Dynamically -- the load-bearing half, since nothing static can tell a collector
that propagates the exit code from one that swallows it -- its `--self-test`
extracts each live block out of lint.yml and runs it as `bash -e <file>`
against stubs with controlled exit codes, reading "did this command run" from
the stub's own side effect rather than from the block's output. The same
command list is also driven through the pre-fix bare sequence, which must mask.
Census behind the narrow rule: 343 `run:` steps across 26 workflows, of which
exactly two carried two or more distinct self-tests, both in lint.yml's `lint`
job and both fixed here. Every other multi-command block is a precondition or
a dependency sequence, where abort-on-first-failure is the correct semantics.
Part of #10814
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
* docs(ci): rest the direct-node invocation on precedent, not on an over-broad #9465 fence
The `Step-collector gate` step comment and the script docblock both claimed
root `package.json` is "declared territory of the @changesets/cli v3 migration
lane (#9465)". That reading is over-broad. The fence's own text scopes it to
that file's `@changesets/cli` range and its `version` script -- the
parenthetical in the issue body is scoping, not illustrative -- so the file
itself is not fenced and a `check:step-collectors` key would have been allowed.
The decision is unchanged and still correct: lint.yml already invokes several
gates as `node scripts/...` directly, and dispatch-gates.mjs derives gate
families from either spelling, so the direct form loses no discovery and adds
no key to the root manifest. Only the stated REASON was wrong.
Worth more than a silent reword, because it is this PR's own defect class one
level up: a comment asserting a constraint nobody verified reads exactly like
a verified one, and propagates to the next agent who reads it -- the same way
a stale knownGap turns into a stale permission. So the accurate scope is now
stated positively next to the decision rather than merely deleted.
Only the two comments I authored are touched; the seven pre-existing instances
of the same phrasing elsewhere in lint.yml are left alone as out of scope.
Part of #10814
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 2695a8c commit ee8f7c3
2 files changed
Lines changed: 642 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
184 | 184 | | |
185 | 185 | | |
186 | 186 | | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
187 | 196 | | |
188 | 197 | | |
189 | | - | |
190 | | - | |
191 | | - | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
192 | 226 | | |
193 | 227 | | |
194 | 228 | | |
| |||
527 | 561 | | |
528 | 562 | | |
529 | 563 | | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
530 | 598 | | |
531 | 599 | | |
532 | | - | |
533 | | - | |
534 | | - | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
535 | 674 | | |
536 | 675 | | |
537 | 676 | | |
| |||
0 commit comments