tools: the merge gate reads a merged PR as green, and a skip can outrank a failure #16046

Two blind spots in scripts/pr_required_gate.py

Both found by using the gate, not by reading it. PR opened alongside this issue.

1. A merged PR reads CONTEXTS-GREEN. Its required contexts all succeeded — before it landed. The verdict is true and useless, and read as clearance it says "ready to merge" about something already merged.

That is not hypothetical: I read it as clearance and told another session a merge window was open, alongside three PRs that had merged hours earlier.

2. A skip can outrank a conclusive failure. The gate takes the newest observation per context name. That is right for supersession — one workflow re-run leaving cancelled behind a success — and wrong for two workflows publishing one context name, where a path-filtered shim can land skipped AFTER a real failure on the same commit.

Observed on #16027: the gate reported green while the merge button stayed red. The tool disagreeing with the thing it exists to predict.

The property underneath, which matters more than either fix

These are the third and fourth states this tool could not see. The others:

• pending contributing to no bucket, so a histogram of pass/fail cannot see a suite that has not started
• a conflicted PR producing no runs at all, which reads identically to "CI has not started"
• a superseded cancelled taken as the current answer

Four symptoms, one design property: a check that enumerates conditions is blind to the conditions it does not enumerate, and every blind spot reads as success.

The corollary is the part that should shape the code rather than sit in an issue: the design has to assume there is a fifth. A verdict that cannot say "I do not know what I did not look at" will always spend its blind spots as green — which is the same failure as a reach floor bound to findings rather than to what was examined.

Acceptance criteria

• A merged or closed PR reports a non-green verdict naming its state, and exits non-zero
• A skip never overrides a conclusive result in either ordering — the API returns runs unordered, and the case where the skip is seen first is the one a naive fix misses
• Supersession still resolves to the newest conclusive observation, so cancelled → success and failure → success are unaffected
• A lone skip is still the answer for a path-filtered context nothing else reported — otherwise "a skip is never authoritative" becomes "path-filtered contexts never report"
• Each fix has a contrast: an OPEN PR with green contexts is still green, and the skip rule does not make the gate refuse everything
• The blind-spot property is stated where a future editor reads it, not only in this issue

Note for whoever extends it next

A mutation that ignored skips entirely passed every test written for finding 2, because all of them put the conclusive run later in the list. The unordered-API ordering is the only path to the promotion branch. Worth assuming the same is true of the fifth blind spot: the test that would catch it is probably one nobody has thought to write, not one that exists and fails.