ci(review-coverage): count a Review-Coverage footer only when the pre-push helper could have written it

[kriszyp]

The review-coverage check now counts a Review-Coverage: footer toward mode: enforce only when it matches the exact grammar the cross-model-review helper (formatReviewCoverage in skills-internal's prepush-policy.mjs) writes. A hand-typed or hand-edited footer the helper could not have produced is still reported, but it counts 0, and the check names the mismatch and points at pr-body-review-need.mjs --write.

The trigger: a PR carried the hand-written footer authored=claude @ 5d0901e86b8f; ran=codex,gemini; rounds=4 @ 5d0901e86b8f, which credited a CI bot's review as the gemini leg, and the check passed with 2 families. structuredCoverage in the vendored reviewGate.mjs parses leniently (split on ;/=, strip @…), so any footer that roughly looks right was counted.

For the human reviewer

1. Requirement: warranted, and scoped as specified. The task asked for strict grammar in the CI-only layer (evaluateCiCoverage.mjs), leaving the vendored reviewGate.mjs byte-identical with HarperFast/dispatch dispatch/lib/reviewGate.mjs. I followed that. The trade-off is that the dispatch server gate still parses leniently: it counts a hand-edited footer this check now rejects, so the two gates can disagree on the same PR body. Porting the same check to dispatch/lib/reviewGate.mjs is the follow-up. Declining it here is cheap to reverse.
2. Exact grammar, not structural checks only. The check rejects any footer the current helper could not write (leg allowlists and segment table): segment order authored; ran; [adjudicated]; [blocked]; [declined]; rounds; [full], known leg names per segment (ran= accepts only coverage legs, so domain/conformance there are rejected), no repeated legs, canonical positive integers, and exactly one trailing @ <12 lowercase hex> pin. The alternative was to check only known keys and the pin, and tolerate unknown legs. Cost of the strict choice: the leg list mirrors REVIEW_LEGS by hand, so a new review leg must land here before the helper emits it, or compliant PRs naming it go red. The README now says so. Loosening later is a one-line change.
3. Fail closed, and the remedy text does not blame the author. A malformed footer counts 0 and reds the check in enforce mode. An author on a stale helper checkout hits the same failure, so the message says "a hand edit, or a helper older than this check" and asks for a current pr-body-review-need.mjs --write. It also covers the opposite case: if a current helper wrote the footer, this check's grammar is behind and needs the new value. The alternative was to count the footer and only warn; switching to that is a one-line change.
4. Shape check, not authentication. A carefully forged well-formed footer still passes. Receipt-backed verification (for example, per-leg commit statuses posted by the review CLI) is a separate design with its own trust root. The README's Known limits states this plainly.
5. Head-pinning is not added here. evaluatePrFormat.mjs already fails a coverage pin that is not the current head (format check), and the coverage README deliberately reports but does not enforce the pin. This check only requires the pin to be well-formed.
6. Producer parity is checked by hand, not in CI. The "helper-shaped" fixtures are written in this repo, because skills-internal is private and cannot be imported here. I checked parity once in scratch against the real formatReviewCoverage and 305 real footers (see Verification). Nothing in this repo catches later drift. A generated fixture file, or a parity test on the skills-internal side, would, and is cheap to add later.
7. Pre-existing, not fixed (vendored file): LEG_FAMILY in reviewGate.mjs has no entry for cursor-kimi/cursor-muse, so both score as the cursor family. ran=cursor-composer,cursor-kimi counts 1 where the producer counts 2 (families kimi, muse). This fails closed (under-count). The fix is two map entries in dispatch's canonical copy, then a re-vendor.

Changes

• evaluateCiCoverage.mjs: new coverageFooterProblem(line) returns '' for a helper-shaped footer, or a phrase naming the first mismatch. evaluateCiCoverage selects the last live footer as before. It runs the grammar on the unmasked body line (masking would hide a trailing <!-- --> or inline code the helper never writes) and zeroes enforceable on a mismatch. That also blocks the Complexity: easy waiver. Reported families still come from structuredCoverage, reading the masked line so a comment suffix cannot rewrite them. A last footer too broken to parse now reports 0 rather than falling back to an earlier footer.
• README.md: two sentences. Only helper-grammar footers are enforceable. The grammar catches malformed edits, not forgeries, and its leg list must track REVIEW_LEGS.
• reviewGate.mjs: unchanged.

Verification

Route: not observable end-to-end before merge. The workflow runs from the base checkout (pull_request_target), so this PR's own check still runs main's evaluator. The evaluator is covered through its real CLI entry point instead.

• node --test .github/actions/review-coverage/*.test.mjs: 127 pass, 0 fail. New cases in ci-review-coverage.test.mjs:
  • Seven helper-shaped variants are enforceable, including through a CRLF body: ran=none, authored=unknown, adjudicated/blocked/declined, claude(fallback)(reason), with and without full=, with and without <sub>.
  • Twenty malformed footers are not enforceable, each with its named reason: the observed double-@ footer, out-of-order segments, unknown and non-coverage legs in ran=, a missing pin, duplicated @, a short or uppercase pin, an unknown segment, a reasonless blocked=, a repeated leg or segment, rounds=01, a missing rounds=, an unpaired <sub>, and comment or inline-code suffixes.
  • Additional cases cover last-footer selection in both orders, the easy-waiver refusal, the empty-last-footer report, the masked-suffix report, and exit status 1 plus the named reason through ci-review-coverage.mjs --mode enforce.
• Parity with the real producer (scratch, not committed, since skills-internal is private):
  • Every Review-Coverage: line from the last 400 HarperFast/harper PR bodies was run through coverageFooterProblem. All 305 helper-produced lines pass. The 5 rejected lines are 4 hand-written footers plus one CRLF artifact:
    • fix(status): reflect component-load failure in the availability the GTM path reads: the observed double-@ footer. It is open and will go red on its next sync, which is intended.
    • fix(components): scope the handleApplication lock per application (merged): a reasonless blocked=gemini. The helper has always written leg(reason).
    • Refuse an engine-only restore over a database that still has blobs: a prose line. It already scored 0 (no authored=), so its check does not change.
    • Keep computed resolver output out of durable records (closed): unavailable= and a 9-hex pin.
    • Keep the redirector integration test off the npm registry fails only when its raw CRLF line is fed in directly. The evaluator normalizes CRLF first, so it passes in CI.
  • formatReviewCoverage from skills-internal origin/main, run on synthetic receipts (fallback leg, blocked/declined, unknown author, full=), produced footers that all pass.
• npx oxlint --deny-warnings .github/actions/review-coverage/ and prettier --check are clean. git diff main -- .github/actions/review-coverage/reviewGate.mjs is empty.

Complexity: medium

🤖 Generated with Claude Code

https://claude.ai/code/session_01Cj4LC7bsgYR6j8FPPG6EUW

— Claude Opus 5.5

Origin — the dispatch brief this PR was written from

Harden the review-coverage GitHub Action (.github/actions/review-coverage/) so a Review-Coverage: footer counts as enforceable coverage only when it matches the exact grammar the cross-model-review footer helper emits. A hand-typed or hand-edited footer that the helper could not have produced must score as not enforceable (count 0 in mode: enforce), with a report line naming the mismatch and pointing at pr-body-review-need.mjs --write.

Acceptance

Tests in .github/actions/review-coverage/ci-review-coverage.test.mjs (or a sibling) covering: every helper-shaped footer variant (ran=none; with adjudicated/blocked/declined; with and without full=; with and without _{) is enforceable; the observed malformed footer above, a footer with segments out of order, an unknown leg name in ran=, a missing trailing @ <sha>, and a duplicated @ are NOT enforceable and the report names why. reviewGate.mjs unchanged. README.md in the action updated in one or two sentences to say only helper-grammar footers are enforceable. Follow harper-engineering-guidelines DLC; draft PR.}

_{Dispatch: task harper-review-coverage-strict-footer · queued by kris-session · ran by claude/opus/high · worker kzyp-xps-1}

_{Review-Coverage: authored=claude; ran=codex,gemini; adjudicated=domain; declined=cursor-grok,cursor-composer,cursor-kimi,cursor-muse; rounds=3; full=1 @ 4df1292}

_{Human-Review-Need: 3 (decisions: exact-helper-grammar, parity-by-hand, ci-only-strictness) @ 4df1292}

[gemini-code-assist]

Code Review

This pull request introduces strict grammar validation for the Review-Coverage: footer in the review-coverage CI action. It adds a new coverageFooterProblem helper in evaluateCiCoverage.mjs to parse and validate the footer's segments (such as authored, ran, and rounds) and its trailing SHA pin against the exact format emitted by the helper tool. If a footer is hand-edited or malformed, it now counts as zero toward enforcement. Comprehensive unit tests have been added to ci-review-coverage.test.mjs to verify various well-formed and malformed footer scenarios, and the README has been updated accordingly. There are no review comments to address, and I have no additional feedback to provide.

[claude]

Reviewed; no blockers found.

[cb1kenobi]

The CI coverage check now counts a Review-Coverage footer toward enforce mode only when it matches the helper grammar, so a hand-edited double-@ pin scores 0. The last footer still wins, families still come from the masked line, and a grammar mismatch also blocks the easy-complexity waiver. Helper-shaped variants, the observed malformed footer, and the CLI enforce path are tested. No confirmed blockers on the changed lines.

—
Reviewed 4df1292