You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The two most expensive PRs of 2026-08-15 were both preceded by the builder judging the change small enough to skip the design pass. Both judgements were wrong, and in both cases the reviewers spent rounds finding what a design pass would have prevented.
PR #610 — 8 review rounds. Scores 3, 6, 4, 8, 8, 9/8, 9/9. Rounds 1–3 were spent modelling codex's option grammar in a regex, one correction at a time (an option's value read as the subcommand; compact short values; terminal -h/-V; variadic --image; per-wrapper arity). Round 4 deleted the grammar and the score jumped to 8 immediately. The design question — "should this predicate know the CLI's option grammar at all?" — was never asked before round 4. It was answerable on day one.
PR #614 — 2 rounds so far. Both priority rulings named #613 as next, and the builder went straight from "it's next" to writing code. Two defects followed:
The probe subshell inherited set -e, so a non-zero gh — exactly what a red build is — killed it before it wrote its status marker. Every red build would have burned the full deadline and reported "abandoned."
Neither is subtle. Both are the kind of thing a design pass surfaces as a question ("what enforces the deadline, and does it exist everywhere this runs?") rather than the kind a reviewer should be spending a round on.
Why the existing rungs do not catch it
The ladder is advisor/planner → builder → reviewer, and the planner rung is real — Fable ruled the #610 redesign and both models ruled the #613 priority. What is missing is that entry to the planner rung is at the builder's discretion, and the builder's own size estimate is the gate. "It's two lines" is not a measured property; it is the judgement of the party least positioned to make it, made before the work exists.
#593 Rung 1's stop condition measures the symptom — "no review round that changes zero behaviour" — but nothing measures the cause. A skipped design pass is invisible until the reviewer bills for it.
Proposal
A design pass is mandatory before any PR, with no size exemption. Concretely, the smallest form:
The handoff's scope_decisions[0] (round 1, decision: "start") must record a design_ruling field: who ruled on the approach, and the one-sentence question they answered. Not "this is next" — the how, not the what.
If no design pass happened, the field says so explicitly, so the omission is visible in the artifact the reviewer reads rather than inferable only from the round count afterwards.
That is one field in a file the gate already reads, and it modifies an existing surface rather than adding a guard — consistent with #593's standing "no new guard or tooling surface" ruling.
Deliberately not proposed: a hook or test that blocks on a missing design ruling. This repo's standing rule is that no test may cost more rounds than the change it guards, and #608 just closed for failing it. Visibility first; enforcement only if visibility proves insufficient.
Acceptance criteria
The handoff contract requires a design-pass record at round 1, with an explicit "none" being a legal but visible value
The review-leg prompt surfaces that field, so a reviewer can see whether the approach was ever ruled on
Demonstrated on a real PR: the field is present and a reviewer references it
Honest uncertainty
This may be a discipline problem that no artifact fixes — the builder that skips the pass can also write "design ruled by: me". The counter-argument is that making the omission visible to the reviewer changes who gets to notice it, which is the same mechanism #613 uses for CI status: the fix was never more scrutiny, it was putting the fact where the reviewer already reads.
Worth ruling on before building. Which is the point.
The pattern
The two most expensive PRs of 2026-08-15 were both preceded by the builder judging the change small enough to skip the design pass. Both judgements were wrong, and in both cases the reviewers spent rounds finding what a design pass would have prevented.
PR #610 — 8 review rounds. Scores 3, 6, 4, 8, 8, 9/8, 9/9. Rounds 1–3 were spent modelling codex's option grammar in a regex, one correction at a time (an option's value read as the subcommand; compact short values; terminal
-h/-V; variadic--image; per-wrapper arity). Round 4 deleted the grammar and the score jumped to 8 immediately. The design question — "should this predicate know the CLI's option grammar at all?" — was never asked before round 4. It was answerable on day one.PR #614 — 2 rounds so far. Both priority rulings named #613 as next, and the builder went straight from "it's next" to writing code. Two defects followed:
timeout, which does not exist on stock macOS including this repo's own machine, justified by a code comment asserting "gh carries its own network timeouts." That claim was never verified. The reviewer checked the source (cli/cliapi/http_client.go, go-ghpkg/api/client_options.go) and gh 2.92 sets no overall HTTP timeout. A stalled request would block beforeexec codexran — codex exec hangs forever on stdin; every cross-model review leg needs < /dev/null #590 reintroduced by the fix for Review legs certify over a red build: make CI status a precondition, not more diff scrutiny #613.set -e, so a non-zerogh— exactly what a red build is — killed it before it wrote its status marker. Every red build would have burned the full deadline and reported "abandoned."Neither is subtle. Both are the kind of thing a design pass surfaces as a question ("what enforces the deadline, and does it exist everywhere this runs?") rather than the kind a reviewer should be spending a round on.
Why the existing rungs do not catch it
The ladder is advisor/planner → builder → reviewer, and the planner rung is real — Fable ruled the #610 redesign and both models ruled the #613 priority. What is missing is that entry to the planner rung is at the builder's discretion, and the builder's own size estimate is the gate. "It's two lines" is not a measured property; it is the judgement of the party least positioned to make it, made before the work exists.
#593 Rung 1's stop condition measures the symptom — "no review round that changes zero behaviour" — but nothing measures the cause. A skipped design pass is invisible until the reviewer bills for it.
Proposal
A design pass is mandatory before any PR, with no size exemption. Concretely, the smallest form:
scope_decisions[0](round 1,decision: "start") must record adesign_rulingfield: who ruled on the approach, and the one-sentence question they answered. Not "this is next" — the how, not the what.That is one field in a file the gate already reads, and it modifies an existing surface rather than adding a guard — consistent with #593's standing "no new guard or tooling surface" ruling.
Deliberately not proposed: a hook or test that blocks on a missing design ruling. This repo's standing rule is that no test may cost more rounds than the change it guards, and #608 just closed for failing it. Visibility first; enforcement only if visibility proves insufficient.
Acceptance criteria
Honest uncertainty
This may be a discipline problem that no artifact fixes — the builder that skips the pass can also write "design ruled by: me". The counter-argument is that making the omission visible to the reviewer changes who gets to notice it, which is the same mechanism #613 uses for CI status: the fix was never more scrutiny, it was putting the fact where the reviewer already reads.
Worth ruling on before building. Which is the point.
Related