You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When an issue is created (or updated) with the dev-lead label plus one or more other labels in the same API call, the dev-lead agent can silently fail to be dispatched. GitHub fires a separate issues:labeled event per label; the dispatch job only acts when the triggering label is dev-lead, while a per-issue cancel-in-progress concurrency group lets a later (non-dev-lead) labeled event cancel the very run that would have acted. Net effect: the workflow reports success, but the agent never runs and no PR is opened.
Observed behaviour (concrete evidence)
While working issue #438 (created via gh issue create --label "dev-lead,bug,ci,automation"):
Four issues:labeled events fired at ~21:10, all sharing concurrency group dev-lead-issue-438.
Runs 27306552438 and 27306552128 completed success but the dispatch job classified intent as:
i.e. they were triggered by the bug/ci/automation label events and correctly skipped.
The other two runs were cancelled by cancel-in-progress — and the run triggered by the dev-lead label was among them, so it never dispatched the agent.
Result after 25 min: issue OPEN, 0 comments, no in-progress label, no branch, no PR. A "successful" workflow that did nothing.
Recovery: cycling the label alone (remove + re-add dev-lead, no competing events) fired run 27308171856 →
This also means such issues are not self-healing: the daily compliance-retrigger sweep only re-triggers issues carrying the compliance-audit label, so a plain dev-lead issue stuck this way stays stuck indefinitely.
Root Cause
Per standards/ci-standards.md:421-428, the issue-intent job is gated and serialised as:
Gate keys on the triggering label (github.event.label.name), not on whether the issue currently has the dev-lead label. So only the single event for the dev-lead label can act.
cancel-in-progress collapses all same-issue events, and the cancellation happens at the workflow/concurrency layer before the if: decides a run is a no-op. So a non-acting run (wrong label) can cancel the one acting run (dev-lead label) purely based on arrival order.
When dev-lead is not the last label applied, the acting run loses the race.
Reproduction
gh issue create --repo <org>/<repo> --label "dev-lead,bug" --title t --body b
Observe the dev-lead runs: the dev-lead-triggered run is cancelled; survivors skip with label-not-dev-lead.
Issue receives no agent dispatch (no plan comment, no PR).
Confirm recovery by cycling dev-lead alone.
Recommended Fix
Gate on the issue's current label set rather than the triggering label, and let per-issue concurrency collapse the duplicate events onto a single run that still acts:
With this, every one of the N labeled events passes the gate; cancel-in-progress collapses them to the last-arriving run, which now acts (because the gate no longer depends on which label triggered it). Pair with an idempotency guard so re-entrant events can't double-dispatch (skip if a dev-lead/issue-<n>-* PR or fresh in-progress label already exists — the same check dl_dev_lead_active already encodes).
Apply the same fix to the org template (standards/workflows/dev-lead.yml / the reusable) and roll it to the fleet.
Acceptance Criteria
Creating an issue in a single API call with dev-leadplus ≥1 other label dispatches the agent exactly once (not zero).
Dispatch does not depend on dev-lead being the last label applied.
Adding dev-lead to an issue that already carries other labels (single add) still dispatches.
At most one concurrent agent run per issue; simultaneous events never produce duplicate PRs.
Label cycling / re-trigger recovery path (relied on by compliance-retrigger.sh) still works.
Regression test simulates multiple near-simultaneous issues:labeled events where dev-lead is not last, and asserts exactly one agent dispatch.
Standard (standards/ci-standards.md) and template updated to match.
Context
Discovered while monitoring #438 (whose fix, PR #442, is unrelated to this race). Part of the Compliance program / dev-lead reliability work. Sibling to #431/#432 (lost-labeled-event recovery).
Note: when assigning this to dev-lead, add the dev-lead label on its own (or cycle it last) to avoid triggering the very race described here.
Summary
When an issue is created (or updated) with the
dev-leadlabel plus one or more other labels in the same API call, the dev-lead agent can silently fail to be dispatched. GitHub fires a separateissues:labeledevent per label; the dispatch job only acts when the triggering label isdev-lead, while a per-issuecancel-in-progressconcurrency group lets a later (non-dev-lead) labeled event cancel the very run that would have acted. Net effect: the workflow reports success, but the agent never runs and no PR is opened.Observed behaviour (concrete evidence)
While working issue #438 (created via
gh issue create --label "dev-lead,bug,ci,automation"):issues:labeledevents fired at ~21:10, all sharing concurrency groupdev-lead-issue-438.27306552438and27306552128completedsuccessbut the dispatch job classified intent as:bug/ci/automationlabel events and correctly skipped.cancelledbycancel-in-progress— and the run triggered by thedev-leadlabel was among them, so it never dispatched the agent.in-progresslabel, no branch, no PR. A "successful" workflow that did nothing.dev-lead, no competing events) fired run27308171856→This also means such issues are not self-healing: the daily compliance-retrigger sweep only re-triggers issues carrying the
compliance-auditlabel, so a plaindev-leadissue stuck this way stays stuck indefinitely.Root Cause
Per
standards/ci-standards.md:421-428, the issue-intent job is gated and serialised as:Two facts combine into the race:
github.event.label.name), not on whether the issue currently has thedev-leadlabel. So only the single event for thedev-leadlabel can act.cancel-in-progresscollapses all same-issue events, and the cancellation happens at the workflow/concurrency layer before theif:decides a run is a no-op. So a non-acting run (wrong label) can cancel the one acting run (dev-leadlabel) purely based on arrival order.When
dev-leadis not the last label applied, the acting run loses the race.Reproduction
gh issue create --repo <org>/<repo> --label "dev-lead,bug" --title t --body bdev-lead-triggered run is cancelled; survivors skip withlabel-not-dev-lead.dev-leadalone.Recommended Fix
Gate on the issue's current label set rather than the triggering label, and let per-issue concurrency collapse the duplicate events onto a single run that still acts:
With this, every one of the N labeled events passes the gate;
cancel-in-progresscollapses them to the last-arriving run, which now acts (because the gate no longer depends on which label triggered it). Pair with an idempotency guard so re-entrant events can't double-dispatch (skip if adev-lead/issue-<n>-*PR or freshin-progresslabel already exists — the same checkdl_dev_lead_activealready encodes).Apply the same fix to the org template (
standards/workflows/dev-lead.yml/ the reusable) and roll it to the fleet.Acceptance Criteria
dev-leadplus ≥1 other label dispatches the agent exactly once (not zero).dev-leadbeing the last label applied.dev-leadto an issue that already carries other labels (single add) still dispatches.compliance-retrigger.sh) still works.issues:labeledevents wheredev-leadis not last, and asserts exactly one agent dispatch.standards/ci-standards.md) and template updated to match.Context
Discovered while monitoring #438 (whose fix, PR #442, is unrelated to this race). Part of the Compliance program / dev-lead reliability work. Sibling to #431/#432 (lost-
labeled-event recovery).