Skip to content

Auto-rebase eligibility gate inverts the review-currency goal (reusable-side scope) #926

Description

@don-petry

Summary

Companion to petry-projects/.github-private#1416, which audits why the auto-rebase automation leaves most of the fleet's open PRs behind main (live snapshot: 50 open non-draft/non-Dependabot PRs across 11 consumer repos, 0 with an APPROVED review, 30 BEHIND, 12 already decayed into unresolvable DIRTY conflicts). Root cause: auto-rebase-reusable.yml's eligibility gate (.github/scripts/auto-rebase/lib/eligibility.sh) only updates a PR's branch after it already has an APPROVED review — inverting the actual goal, which is to keep PRs current so a reviewer can approve them with confidence.

Full analysis, fleet evidence, and the parts of this scoped to .github-private (ruleset config, repo-template's channel-pin, AGENTS.md doc drift) live in #1416. This issue tracks only the subset of acceptance criteria that require changes to this repo — the reusable workflow and its scripts.

Decisions (2026-08-02) — both blocking open questions resolved

  • Fan-out control: accept the CI cost. Default eligibility flips from
    review-ready to all — this mode already exists in lib/eligibility.sh, so it's a
    default-value change, not new logic. The org's own decision record
    (docs/initiatives/auto-rebase-vs-merge-queue.md §5 in .github-private) measured
    ~50s CI per forced re-test and <0.5% of serial-merge capacity used — the original
    170–220-runs/day estimate behind the review-ready gate (Restrict auto-rebase fan-out to review-ready PRs (free, plan-independent mitigation) #465/[Fleet Monitor] petry-projects/.github — .github/workflows/canary-rollout.yml #738) was the
    pre-optimization worst case, not today's reality, and it directly works against the
    actual goal (every PR stays mergeable, not just approved ones).
  • AC2 contingency: if the approval-survival spike (below) confirms the dismissal
    happens, implement an actor-scoped bypass for the auto-rebase automation identity
    on the pr-quality ruleset, not a fleet-wide relax. Unverified caveat: GitHub
    rulesets scope bypass actors at the ruleset level, not per-rule — confirm whether an
    actor can be exempted from just dismiss_stale_reviews_on_push/
    require_last_push_approval while the rest of pr-quality still applies to them. If
    that's not supported, do not silently fall back to the fleet-wide relax — flag
    dev-lead:needs-human so a human picks the fallback explicitly.

Scope — reusable-side acceptance criteria

  • Proactive currency (#1416 AC1). Flip the reusable's eligibility input
    default from review-ready to all per the decision above.
  • Pre-flight approval-survival spike (#1416 AC2). On a real test PR: approve it
    while behind, let auto-rebase run update-branch(update_method=merge), and check
    whether the approval survives (timeline: any review_dismissed event, or
    reviewDecision flipping off APPROVED). Config context: 9 of 11 consumer repos
    have both dismiss_stale_reviews_on_push: true and require_last_push_approval: true on their pr-quality ruleset — verified at the config level, but whether a
    merge-method update-branch push actually triggers the dismissal is
    unconfirmed, and was unobservable fleet-wide at analysis time (0 approved open
    PRs to test against). This is a verification spike, not an assumed defect. If
    confirmed real, implement the actor-scoped-bypass decision above (ruleset-side,
    .github-private + fleet scope) with its documented fallback; if not confirmed,
    close with the evidence.
  • Merge method as a locked invariant. update-branch already correctly uses update_method=merge (never rebase) — add a regression test (bats or equivalent) asserting this so it can't silently regress.
  • Conflict routing verification. Once the eligibility change lands, confirm a DIRTY PR is actually attempted by update-branch and therefore reaches the existing <!-- auto-rebase-conflict: --> sentinel → dev-lead path (this mechanism already works correctly for eligible PRs today — the bug is only that ineligible-but-conflicted PRs never reach it). If confirmed, retire the weaker one-time <!-- auto-rebase-stuck --> escalation (issue auto-rebase deadlock: 'review-ready' gate skips unapproved PRs but the auto-rebase:ready label is never applied (#465 gap) #711 in .github-private) as redundant.
  • Regression coverage. A behind, conflict-free, non-draft PR — approved or not — gets updated by auto-rebase, and if it was approved beforehand, remains approved/mergeable afterward (enforces the AC2 spike's finding rather than just having observed it once). Cover in the existing bats suite for lib/eligibility.sh / the reusable's update logic.
  • Remove the dead escape hatch. Once the eligibility change ships, delete the auto-rebase:ready label's handling and the eligibility input's review-ready mode (or the whole input, if nothing else consumes it) from the reusable and lib/eligibility.sh — its only purpose was routing around the approval gate, which no longer exists. (The label definition itself and any doc references are consumer-repo-side cleanup, tracked in #1416.)

Release note

Per this org's versioning convention (docs/release/versioning.md in .github-private), this is a behavioral change to the reusable and should ship as a new release + channel promotion (cut-release.sh auto-rebase <version> --channel next, promoted through the ring sequence), not a silent edit to an existing tag. .github-private#1416 AC6 covers fixing repo-template's orphaned auto-rebase/stable pin so this eventually reaches newly-scaffolded repos too — that fix should land before or alongside this one, not after, or new repos keep inheriting the pre-fix behavior indefinitely.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsinitiativeinitiative:autoArms an initiative epic for the initiative-driver auto-release

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions