Skip to content

mergeability: the behind condition is invisible whenever mergeable_state outranks it (blocked, unstable, draft) #463

Description

@zendev-acceptor

Goal

Make the required mergeability check actually fail on the behind condition it documents, including when GitHub's mergeable_state is reporting some other, higher-precedence word for the same pull request.

Evidence

Observed on PR #459 at head 3545c375f558265b84a5538219864930ee18e1f7 during an ACCEPTOR run on 2026-09-14:

$ gh api repos/drevendev/trade_simulation/pulls/459 --jq '{mergeable, mergeable_state}'
{"mergeable":true,"mergeable_state":"blocked"}

$ gh api repos/drevendev/trade_simulation/compare/3545c375...master --jq '{ahead_by, behind_by}'
{"ahead_by":4,"behind_by":3}          # master has 4 commits the branch does not

$ gh pr checks 459 | grep mergeability
mergeability   pass   merges cleanly into the base branch (blocked)

The branch was behind its base by four commits (a52535e, 8a89f5d, 5a56eb1, 19e6c65) and mergeability reported success.

Cause: scripts/mergeability.py::classify() decides behind from mergeable_state == "behind". mergeable_state is a single precedence-ordered word, and dirty > draft > blocked > behind > unstable. So whenever a pull request is also blocked (a standing CHANGES_REQUESTED, an unmet required review) or also unstable (a failing non-required check), GitHub never says behind and the condition is silently invisible.

That is not a rare corner. It is exactly the population an ACCEPTOR re-reviews: a pull request carrying a standing change request is blocked by definition, so for every rework round the behind half of this check is switched off. The same masking also disables the forge's own repair — scripts/update_branches.py::should_update() deliberately skips blocked ("dirty, clean, blocked and an uncomputed answer are all left exactly as they are"), so a behind-and-blocked branch is neither gated nor updated by anything.

The documented contract says otherwise, in two places:

  • scripts/mergeability.py docstring / STALE_BASE comment: behind is "reported as a failure rather than left to branch protection's 'require branches to be up to date'";
  • docs/zendev/ACCEPTOR_RUNBOOK.md section 3: "It fails on two conditions, not one. ... A branch that is merely behind merges cleanly and is still refused."

A reviewer who follows the runbook and reads the check, as the runbook tells it to, gets a green answer to a question the check did not measure. On #459 the gap was harmless — the four commits touched docs/spec/mirror/**, docs/zendev/schemes.json and scripts/schedule_watchdog.py, none of which the pull request touches, and the ACCEPTOR closed it by merging current master locally and re-running both suites green. Two changes that were semantically incompatible would have landed unnoticed, which is the precise failure this check exists to catch.

Scope

  • scripts/mergeability.py: determine "behind" from a source that does not get masked — e.g. GET /repos/{repo}/compare/{head}...{base}'s behind_by, or compare/{base}...{head}'s behind_by/status — rather than from mergeable_state alone. Keep mergeable is None → pending; keep mergeable is False → failure.
  • scripts/tests/test_mergeability.py: extend the classify() (or its successor's) table with the masked combinations — behind + blocked, behind + unstable, behind + draft.
  • scripts/update_branches.py: decide whether should_update() should act on the same unmasked signal. A branch that is behind and blocked is repairable by exactly the merge this script performs; leaving it to the author is a choice, but it should be a stated one rather than a consequence of the same masking.

Non-goals

  • No change to branch protection settings, and no reliance on "require branches to be up to date": the reasoning in mergeability.py for preferring a red check over that setting still holds — the setting blocks a merge while producing no check, so the work belongs to no queue.
  • No change to the dirty/conflict path, which is unaffected.
  • No change to docs/zendev/ACCEPTOR_RUNBOOK.md. The runbook already describes the intended behaviour; this is the implementation catching up, not a policy change.
  • No simulation or src/** change.

Acceptance criteria

  1. A pull request that is behind its base and whose mergeable_state is blocked receives a failure mergeability status naming the stale base.
  2. The same holds for unstable, and for any other mergeable_state that outranks behind.
  3. A pull request that is genuinely up to date and mergeable still receives success; an uncomputed answer still receives pending, never success.
  4. scripts/tests/ covers all three masked combinations from criterion 2 as pure classifier cases, with no network access.
  5. Whatever is decided for scripts/update_branches.py is implemented and its docstring states the decision explicitly, so a later reader does not have to re-derive it.
  6. Required TypeScript and retained .NET gates remain green.

Verification

python -m unittest discover -s scripts/tests, plus npm run typecheck, npm test, npm run build, dotnet build --configuration Release --no-restore and dotnet test --configuration Release --no-build.

Type: bug
Area: tooling
Priority: normal


Opened by an ACCEPTOR run reviewing PR #459, per ACCEPTOR_RUNBOOK section 1 ("a selector that picks the wrong pull request is a defect in the control plane, and the run that notices is the only thing that can report it") and section 2 ("an observed failure is what moves a gate"). Not folded into #459: it is a control-plane change and not that pull request's scope.

Activity

  1. added
    type:bugVerified behavior differs from the intended contract
    area:toolingCI, scripts, guards, developer tooling
    status:readySpecified and unblocked; safe for an agent to claim
    policyControl-plane change: workflows, scripts, runbooks, AGENTS.md. Operator-owned, never AUTHOR.
    on Sep 14, 2026
  2. drevendev commented on Sep 30, 2026

    @drevendev
    Owner

    Closure — fixed before this Issue was picked up

    Both halves were repaired on 2026-09-15, after this Issue was filed:

    Seen working on #767 on 2026-09-29/30: the required status read "behind master by 1 commit" for as long as the branch was behind. Checked on master 94dcec3b.

  3. removed
    status:readySpecified and unblocked; safe for an agent to claim
    on Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:toolingCI, scripts, guards, developer toolingpolicyControl-plane change: workflows, scripts, runbooks, AGENTS.md. Operator-owned, never AUTHOR.priority:normalDefault planned prioritytype:bugVerified behavior differs from the intended contract

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions