Repository navigation
fix: the gate job had no headroom, so a slow run read as a failure - #235
Merged
Merged
Conversation
#228's gate ran 10m09s against timeout-minutes: 10 and was reported CANCELLED. I first blamed my own branch pushes for cancelling it through the concurrency group; that was wrong, and the timestamps say so. The two most recent merges ran this job for 8m53s and 9m10s — 89% and 92% of its budget. That is not a flaky run, it is a budget with no room in it, and the next slow semgrep crosses the line whatever the change was. A timeout surfaces as "cancelled", which at the merge gate is indistinguishable from a real failure. Worse than a red build: a red that sends somebody hunting a defect that does not exist, in a repository whose rule is that a check which could not run is never reported as one that passed. The inverse deserves the same care. Twenty minutes, with the measurement written next to the number so the next person changing it knows what it was set from.
piwi3910
force-pushed
the
fix/gate-timeout-headroom
branch
from
August 27, 2026 12:05
3b45935 to
db8f517
Compare
piwi3910
added a commit
that referenced
this pull request
Aug 27, 2026
787 tracked files, 510 of them markdown, one formatter subprocess each, run sequentially. prettier costs ~0.2s to start and the work per file is trivial, so the pass was almost entirely process startup serialised. checkAll fans the per-file check across NumCPU workers, bounded because every unit of work is an external process and a tree this size would otherwise fork hundreds at once. WHAT IT ACTUALLY BUYS, measured both places rather than assumed: Locally, on ten cores: 257.5s to 153.0s, a 41% cut, same verdict — 772 clean, 0 unformatted, 0 unchecked, 15 out of scope. In CI: no measurable change. That step ran 508s, 407s and 402s on the three most recent main runs, and 429s on this branch. That is inside the existing spread, so this does NOT speed up the gate job and must not be cited as having done so. The runners have far fewer cores, and whatever bounds that step there is not per-file startup. Merged on the local benefit, which is where a developer waits on it, and because the sequential loop had no reason to be sequential. The CI budget was fixed by #235 raising the timeout, not by this. Order is the whole risk. The gate prints findings straight from this slice, so a version appending results as they finished would make the same tree print a different report twice. Results are written by index, and TestCheckAllPreservesTheOrderGiven shuffles file sizes so completion order and given order disagree; the append-under-mutex mutation was applied and watched to fail. Race detector clean over the package. The scope guard moved out of the loop, where it always belonged: it never read the file, so it broke on the first iteration or never. Verifying this surfaced something unrelated, filed as #236: the advisory lint findings come back in a different order on identical runs of the UNCHANGED binary, and the set is capped, so which ones survive the cap varies between runs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#228's gate ran 10m09s against timeout-minutes: 10 and was reported
CANCELLED. I first blamed my own branch pushes for cancelling it through
the concurrency group; that was wrong, and the timestamps say so.
The two most recent merges ran this job for 8m53s and 9m10s — 89% and 92%
of its budget. That is not a flaky run, it is a budget with no room in it,
and the next slow semgrep crosses the line whatever the change was.
A timeout surfaces as "cancelled", which at the merge gate is
indistinguishable from a real failure. Worse than a red build: a red that
sends somebody hunting a defect that does not exist, in a repository whose
rule is that a check which could not run is never reported as one that
passed. The inverse deserves the same care.
Twenty minutes, with the measurement written next to the number so the next
person changing it knows what it was set from.