fix(ci): sweep open PR heads for the infrastructure-retry watchdog (#15139)

[mrveiss]

Thinking Path

PR #15396 added pipeline-scripts/ci_dispatch_retry.py to re-dispatch red checks that were caused by an empty runner pool rather than a real test failure. Re-verified against the merged tree at dfa525dc19:

• HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} is only evaluated in the "Re-dispatch infrastructure-caused reds" step, which is gated if: github.event_name != 'pull_request'.
• On every event that step actually runs on (schedule, push, workflow_dispatch), github.event.pull_request is unset, so HEAD_SHA always fell back to github.sha — the base branch tip, never a red PR's head.
• ci_dispatch_retry.py's run(sha, dry_run) classified exactly that one sha and had zero references to open_pull_requests.

Net effect confirmed: the scheduled sweep re-dispatched checks for Dev_new_gui's own tip on every run and never touched an actual open PR. The sibling ci_dispatch_watchdog.py --check dispatch sweep already solves the "reach every open PR" problem via api.open_pull_requests(base) + collect_heads, so this PR reuses that path instead of inventing a second one.

What Changed

• pipeline-scripts/ci_dispatch_retry.py: run() now lists every open PR against the base branch (WATCHDOG_BASE_BRANCH, default Dev_new_gui, imported from ci_dispatch_watchdog.DEFAULT_BASE_BRANCH), classifies each head via the existing classify_commit, and flattens the infrastructure-caused reds from every PR into one candidate list before calling redispatch(). The --sha CLI flag is gone; --base is optional and defaults to the env var / constant.
  • Admission control preserved and now sweep-wide by construction: budget_blocked (unreadable/low REST rate-limit) and capacity_blocked (in-progress ceiling) are each checked exactly once, before any PR is listed, so a busy queue or an unreadable rate limit stops the sweep before it spends anything. redispatch()'s max_reruns budget walks the single flattened candidate list built across ALL open PRs — once it is spent partway through PR Docs improvements #1's reds, PR Docs improvements #2 (and beyond) are never even reached for re-dispatch, because the counter is never reset per PR. max_attempts (one retry per run) and the per-red job resolution are unchanged.
• .github/workflows/ci-dispatch-watchdog.yml: the "Re-dispatch infrastructure-caused reds" step drops the now-unused HEAD_SHA computation and passes WATCHDOG_BASE_BRANCH: Dev_new_gui instead (matching the "Sweep open PRs" step above it). Event gating (!= 'pull_request') and step ordering (after the self-modification guard) are unchanged, so a PR's own unreviewed copy of this code still never runs with actions: write.
• repo_tests/ci_dispatch_retry_test.py (new file — ci_dispatch_watchdog_test.py is at its frozen line ceiling per instructions, so this doesn't touch it): 5 tests with a fake GitHubApi.request transport, each proven by mutation (see below).

Verification

python3 -m pytest repo_tests/ci_dispatch_retry_test.py -q → 5 passed. repo_tests/ci_dispatch_watchdog_test.py (113 tests, untouched) and repo_tests/python_file_size_ratchet_test.py (31 tests) still pass. Pre-push hook ran the new test file and passed. All functions in the changed module are ≤30 lines (verified via ast); file is 276 lines, well under the 600-line cap.

Mutation evidence (defect reintroduced → test fails → reverted, diff against saved-clean copy confirmed empty after each):

Test	Mutation	Result
test_budget_exhausted_mid_sweep_stops_before_the_next_pr	if done >= limits["max_reruns"]: → if False: (budget check removed)	FAILED — 2 reruns fired instead of 1
test_capacity_at_ceiling_declines_the_sweep	if count >= ceiling: → if count > ceiling:	FAILED — sweep proceeded to list PRs at the ceiling
test_unreadable_rate_limit_declines_rather_than_proceeding	rate_limit_remaining's status != 200 fallback: return -1 → return 999999	FAILED — sweep proceeded as if it had headroom
test_run_attempt_two_or_more_is_left_red_not_retried	if attempt >= limits["max_attempts"]: → if attempt > limits["max_attempts"]:	FAILED — attempt-2 run got re-dispatched
test_sweep_reaches_an_open_prs_head_sha_not_a_base_tip	heads = collect_heads(...) → heads = [] (reintroducing the pre-fix single-sha bug)	FAILED — nothing re-dispatched, proving the open-PR sweep is what reaches the head

Model Used

Claude Sonnet 5

Closes #15139

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.