Repository navigation
[Bug]: watch_pull_request can miss late required gates that pass between polls #15362
Description
Activity
Note
Grok responding on behalf of Julius.
Triage
Thanks for the write-up and for running the evaluator directly, @ryuudotgg! This is a real bug on current
main(5e35272fda), and I didn't find a duplicate or a fix PR. The PR watch added in merged #15057 reports "required checks passed" from a single true/false flag. Because of that, a required check that first shows up already green gets dropped.What happens
evaluatePullRequestWatchinapps/server/src/orchestration-v2/pullRequestWatch.ts(around line 63) treats the required set as passed when every required check is settled and none failed. It emitschecks-passedonly when that's true andwatch.passedis false, then storespassed: true. That flag resets only when the head SHA changes, and it doesn't remember which required checks that report covered.That matches your three evaluations exactly:
- Only
Testsis required, so the evaluator emitschecks-passedwithcount: 1and setspassed. Smoke Tests Gateappears required and already successful, butpassedis already true, sochangesstays empty.- When the new gate is seen pending first,
passedclears, and the next successful read emitscount: 2.
PullRequestWatchReactorsweeps once a minute, so a gate job that GitHub creates after its dependencies and that finishes in a few seconds fits between passes. YourInstall Smoke Gatewindow (created21:33:17Z, successful21:33:22ZUTC) is that case. The earlier wake isn't revised, and the agent was told to end its turn, so nothing resumes it. As you said, this isn't about optional checks or server-side readiness, which #15057 left to the agent.Likely fix area
- One option is to remember the required check names covered by the last pass, like
failedChecksremembers names.checks-passedwould fire again when the settled required set includes a new name, even one that's already successful.ThreadPullRequestWatch.passedinpackages/contracts/src/threadPullRequest.tsis only a boolean, so this would mean a new contract field. - Existing
passed: truewatches without a name list could adopt the current required names on upgrade without a wake, so healthy watches don't all fire once. - A test next to "reports passed once the required checks pass" in
pullRequestWatch.test.tscould lock in both sequences you ran.
A required check with the same name that reruns and passes entirely between two polls would be a separate gap. Until a fix lands, manually resuming is the right workaround. A maintainer will decide on the fix direction.
- Only
Before submitting
Area
apps/server
Steps to reproduce
The following sequence reproduces the problem directly in
evaluatePullRequestWatch, without starting a live watcher:passed: falseagainst a PR containing:Tests: required, successful.Smoke Tests: not required, pending.checks-passedand returns a watch withpassed: true.Tests: required, successful.Smoke Tests: not required, successful.Smoke Tests Gate: a newly appearing required check, already successful.changesarray.This models a dependent required job being created and completing between polling passes. The watcher never observes that job pending.
Expected behavior
When a new required check appears, its successful completion should not be suppressed by the earlier notification for a smaller set of required checks.
The agent should receive an updated required-checks-passed notification even if the new check completed before the next poll.
This report does not request notifications for every optional check or ask the server to determine workflow readiness.
Actual behavior
No new notification is generated.
In
apps/server/src/orchestration-v2/pullRequestWatch.ts, the successful-check notification depends onpassedNow && !passed. Both values remain true when an additional required check first appears already successful.I executed the actual evaluator and reproduced this result. As a control, observing the new gate pending before observing it successful does produce another notification.
The reported live incident involved
ryuudotgg/forge#605, head43fe2891e7ce6525818fc04b99677e63e3b74741:Install Smokecompleted successfully at2026-10-03T21:33:17Z.Install Smoke Gatewas created at2026-10-03T21:33:17Z.2026-10-03T21:33:22Z.The five-second gate lifetime can fall between polling passes. Its timestamps and required status were verified, but the watcher's exact polling history for that incident was not captured.
Impact
Major degradation or frequent failure
Version or commit
0.0.46-nightly.20261003.2632
Environment
No response
Logs or stack traces
{ "initialRequiredPass": [ { "kind": "checks-passed", "count": 1, "required": true } ], "newRequiredGateAlreadySuccess": [], "requiredGateObservedPendingThenSuccess": [ { "kind": "checks-passed", "count": 2, "required": true } ] }Screenshots, recordings, or supporting files
No response
Workaround
Manually resume the agent after the remaining checks finish.