Skip to content

fix(guards): a rationale may open with an issue reference (#16104) - #16107

Closed
mrveiss wants to merge 2 commits into
Dev_new_guifrom
issue-16104-batching-heading
Closed

mrveiss wants to merge 2 commits into
Dev_new_guifrom
issue-16104-batching-heading

Conversation

@mrveiss

@mrveiss mrveiss commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Closes #16104

Thinking Path

scripts/check_pr_issue_batching.py:110 treated any line beginning with # as the next markdown heading:

if stripped.startswith("#"):
    return None

So a ## Single-issue rationale section whose prose opened #15961 alone. The two findings differ. read as empty, and the gate went red on a section that was filled in. An ATX heading is 1–6 # followed by a space, a tab, or end of line; #15961 is an issue reference.

The stop-at-next-heading behaviour is correct and stays — an empty section must not borrow the next section's prose, which is how a heading-only body would pass a check about whether a human justified something. The fix is to match headings properly, not to loosen the scan.

This is #16050 one layer in

That issue records two sessions independently writing the heading form the checker refused — one of them the critical auth fix. The heading form was then accepted, and the natural way to fill it still failed. The gate has now taught a form it refuses, twice.

Why it survived a fix aimed straight at it is the part worth keeping. The hint said "add a section" to an author who had added one, so the only available response was to reword until green. A reword leaves no trace. Every author who hit this fixed it silently, which is why the recurrence looked like a first occurrence.

What Changed

_ATX_HEADING ^#{1,6}(?:[ \t]|$) — replaces startswith("#")
_heading_rationale returns (heading found, prose, the line that ended the section)
_rationale_failure picks the message from those three facts

The message is the second half of the fix, and the one that stops the next recurrence:

The `## Single-issue rationale` section is present but reads as empty. The first
line under it is `## What Changed`, which parsed as the next heading, so the
section ended before any prose was found.

Verification

Run as a known negative first — the new tests were executed against the parser on origin/Dev_new_gui, where they fail:

OLD parser, the defect case                  -> None
OLD parser, check() passes?                  -> False
OLD parser, empty-section message is generic -> True

With the fix, all four of the reporting session's reproduction cases behave, and the two that must keep returning None still do:

THE DEFECT     '#15961 alone. The two findings differ.'  -> counts
control        'Issue #15961 alone.'                     -> counts
must stay None '## Next section'                         -> None
must stay None '# Heading'                               -> None

39 tests pass. # alone is covered too — a bare # is a valid empty ATX heading, #x is not a heading at all.

This PR body is the end-to-end proof. Its rationale below opens with an issue reference — the exact form that was rejected. The workflow checks out the PR's merge ref, so the gate that runs on this PR is the parser this PR ships. If the fix is wrong, this PR goes red on itself.

Model Used

Claude Opus 5 (1M context).

Single-issue rationale

#16104 alone. The other work in flight is a security-gate allowance (#16103), a CI trigger change (#16098) and a process doc (#16102) — different files, different risk, and batching a guard fix with a security allowance would put two unrelated review questions in one pass.


A correction this PR encodes. I had been telling four sessions today that a too-narrow exemption is self-correcting because it goes red. It goes red — that is not the same as going red informatively, and a red with a misleading cause is repaired by working around it rather than by fixing it. Reported by the session that hit the gate, whose reproduction is used verbatim as fixtures.

`check_pr_issue_batching.py:110` treated any line beginning with `#` as the next
markdown heading, so a `## Single-issue rationale` section whose prose opened
`#15961 alone. The two findings differ.` read as EMPTY and the gate went red on a
section that was filled in.

An ATX heading is 1-6 `#` followed by a space, a tab, or end of line. `#15961` is
an issue reference. The stop-at-next-heading behaviour is correct and stays --
an empty section must not borrow the next section's prose -- so the fix is to
match headings properly rather than to loosen the scan.

THIS IS #16050 ONE LAYER IN. That issue records two sessions independently
writing the heading form the checker refused, one of them the critical auth fix.
The heading form was then accepted, and the natural way to FILL it still failed.
The gate has now taught a form it refuses, twice.

Why it survived a fix aimed straight at it is the more useful half. The hint said
"add a section" to an author who had added one, so the only available response
was to reword until green. A reword leaves no trace: every author who hit this
fixed it silently, which is why the recurrence looked like a first occurrence.

So the second change is the message. "No section found" and "section found but
empty" want opposite fixes, and the gate now says which, quoting the line that
ended the parse:

  The `## Single-issue rationale` section is present but reads as empty. The
  first line under it is `## What Changed`, which parsed as the next heading, so
  the section ended before any prose was found.

That is the correction I owe the board today. I had been telling four sessions
that a too-narrow exemption is self-correcting because it goes red. It goes red.
That is not the same as going red INFORMATIVELY, and a red with a misleading
cause is repaired by working around it rather than by fixing it.

Verified as a known negative before trusting it -- the new tests were run against
the parser on base, where the defect case returns None and the empty-section
message is the generic hint. Both now behave, and 39 tests pass.

Reproduction supplied by the session that hit it, and used verbatim as fixtures,
including the two cases that must keep returning None.
@coderabbitai

coderabbitai Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 505d87da-0908-400a-8303-36e290880fdb


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

@mrveiss

mrveiss commented Sep 9, 2026

Copy link
Copy Markdown
Owner Author

Review: approved. The regex is the correct CommonMark rule, the contrast set stops the fix over-correcting, and leaving #16117/#16119 red as evidence is the right call.

The rule

_ATX_HEADING = re.compile(r"^#{1,6}(?:[ \t]|$)")

That is the ATX definition, not an approximation of it. Checked the edges:

  • #15961 alone… — # then 1, no space/tab/EOL → prose, which is the bug
  • ####### seven — #{1,6} consumes six, next char is # → not a heading, matching CommonMark
  • # alone → matches via $ → a heading, also correct

The contrast set is what makes it safe

A fix for "too many things are headings" fails in the other direction if nothing tests it. This has both:

test_a_rationale_opening_with_an_issue_reference_counts     <- the bug
test_a_real_heading_still_ends_the_section                  <- the over-correction
test_a_single_hash_heading_still_ends_the_section
test_a_bare_hash_is_a_heading_not_prose

Without the last three, ^# could have been dropped entirely and an empty section would then borrow the next section's prose as its rationale — which is the failure the original _rationale_under_heading docstring was written to prevent. The fix preserves that property and proves it.

test_the_same_reason_worded_around_the_defect_still_counts is the subtle one: it pins the workaround as well as the bug, so the reworded forms authors were forced into keep working rather than becoming a second class of failure.

The hint is the half that made this survive

Splitting "no section" from "section present but reads as empty" is what closes the loop. The old hint told an author who had added the section to add one — so their only path to green was rewording until the gate stopped complaining, and a reword leaves no trace. That is why this recurred after #16050 and read as a first occurrence to the next person who hit it.

A gate whose remedy text does not match the failure teaches a superstition instead of a fix.

On leaving #16117 and #16119 red

Right, and for the reason you gave: "if a reword leaves no trace, so does mine." Two PRs with a filled ## Single-issue rationale failing the gate are the only durable evidence the defect existed, and they double as an end-to-end test — both should clear on their next run without either being touched. If they do not, the fix is incomplete and that is worth knowing before it is trusted.

I will check exactly that after this merges rather than assuming it.

Verdict

Approved. My gate shows 1 unfinished; merging as soon as it reports.

mrveiss added a commit that referenced this pull request Sep 9, 2026
…16104, #16122) (#16129)

* fix(ci): clear the three defects every open PR is inheriting (#16089, #16104, #16122)

Eight red check-runs across nine PRs reduce to three causes, none of which any of
those PRs introduced. Fixing them one per PR means three CI suites and three
review passes to unblock a queue that is already blocked; fixing them together
means one.

1. DEPENDENCY SECURITY SCAN -- js-yaml ~4.3.1 -> ~4.3.2.
   The advisory range is 4.0.0-4.3.1, so 4.3.2 is outside it and IS the fix. With
   the redocly allowance already on base, this is the half that clears the scan.
   Affects #15980, #16088, #16123.

2. PYTHON-SUITE SHARD 7/12 -- a comment citing a line number.
   comment_line_number_citations_test forbids it, and security.yml carried
   `security_scan_gate.py:100` from a merge earlier today. Mine. A line number is
   invalidated by edits to code it does not describe and fails silently into
   whatever later occupies it -- and it was written inside a paragraph warning
   that the allowance is coarser than it looks. Symbol name only now.
   Affects #15980, #16106, #16107, #16117, #16125.

3. CHECK SAME-SCOPE BATCHING -- an ATX heading needs `#` followed by a space.
   check_pr_issue_batching.py treated any line starting with `#` as the next
   heading, so a rationale opening `#15961 alone...` made a FILLED section parse
   as empty. The message now also distinguishes "no section found" from "section
   found but empty", quoting the line that ended the parse -- because a red only
   self-corrects when it names its real cause, and every previous author here
   reworded until green, leaving no trace.
   Affects #16117, #16119, #16123, #16127.

TAKING DEPENDABOT'S WHOLE package.json WOULD HAVE REGRESSED SIX PACKAGES, and
that is the reason this is assembled by hand rather than merged from its branch.
#16088 was cut before several upgrades landed, so adopting the file wholesale
downgrades apexcharts 7.1.0->6.10.0, cytoscape 3.34.2->3.34.1, onnxruntime-web
1.29.0->1.26.0, vega-embed 7.2.0->7.1.0, vue-i18n 11.4.10->11.4.8 and vue-router
5.3.1->5.2.0. Caught by reading the diff rather than the intent.

So the frontend change is exactly one line of package.json and three lines of the
lockfile -- version, resolved and integrity for node_modules/js-yaml, copied
verbatim from dependabot's verified entry rather than invented. The nine
remaining `4.3.1` strings in the lockfile are debug, @types/esrecurse, ansis and
polished; dependabot's own lockfile has the same nine.

44 tests pass: the batching parser's full suite including the four reproduction
cases, and the citation guard that shard 7 was failing on.

* security: drop the redocly allowance -- the scanner stopped reporting it (#16089)

The gate failed on the allowance itself:

  allowance(s) ['@redocly/openapi-core'] name findings the scanner no longer
  reports. Remove them; an allowance that matches nothing tolerates whatever
  appears next.

Not the advisory returning. The advisory GOING. This is the self-cleaning
property that the allowance was justified against, arriving and doing exactly
what it was described as doing -- and it is the third time today that a check I
verified has then been right about my own work.

Worth being precise about why an empty match is a failure rather than a shrug. An
allowance keyed on a package name admits ANY future advisory in that package, so
one that currently matches nothing is not inert -- it is a standing exemption for
whatever appears there next, with nobody watching. Nothing else in this system
asks whether an allowance is still needed, so without this check the entry sits
until someone happens to reread the block.

The npm gate now carries NO allowances, which is the state to defend. Adding one
back needs the evidence the last one had: no reachable fix, checked against the
REGISTRY rather than the advisory range -- a fix at a major behind a third party's
dependency range is still a fix, which is the distinction that nearly made the
js-yaml allowance permanent -- and the package does not ship.

#16122's subject was pointing the allowance at a removal tracker. There is no
allowance to track, so #16123 is now moot as well as superseded.
@mrveiss

mrveiss commented Sep 9, 2026

Copy link
Copy Markdown
Owner Author

Closing: fully superseded by #16129 (56b75519bc). Verified against merged base, not against the PR description.

Both halves landed. The parser fix, scripts/check_pr_issue_batching.py:94-101 on origin/Dev_new_gui:

# #16104: an ATX heading is 1-6 `#` followed by a space, a tab, or end of line.
# `#15961` is an ISSUE REFERENCE. Treating any leading `#` as a heading made a
_ATX_HEADING = re.compile(r"^#{1,6}(?:[ \t]|$)")

used at :120 to terminate the rationale section. And all eight of this PR's tests are present on base by name:

test_a_rationale_opening_with_an_issue_reference_counts          1
test_the_same_reason_worded_around_the_defect_still_counts       1
test_a_real_heading_still_ends_the_section                       1
test_a_single_hash_heading_still_ends_the_section                1
test_a_bare_hash_is_a_heading_not_prose                          1
test_a_missing_section_gets_the_how_to_add_one_hint              1
test_an_empty_section_says_so_and_quotes_the_terminating_line    1
test_a_section_with_nothing_at_all_beneath_it_says_so            1

I checked the tests individually rather than trusting the merge, because a superseding PR is exactly where a test quietly fails to make the trip — the fix is what gets reviewed, and the cases proving it are what get dropped. 34 tests on base now.

Verified in production, on a PR that was never part of the argument. #16130 was created 11:00:58Z, before this fix merged at 11:27:02Z, and its body has never been edited (lastEditedAt: null). Its rationale opens with #16110. Check same-scope batching now reports success on it, and the job log shows the check reading the section rather than skipping it:

Single issue (#16110), rationale given: #16110 alone, and deliberately not batched
with its sibling. #16111 — the reconstruction test proving a compression is ...

That distinction mattered: the check run publishes no output.summary, so a green is equally consistent with "parsed correctly" and "never reached the section". The log is what separates them.

This PR is CONFLICTING against base because base now contains its own change. Closing rather than rebasing — there is nothing left in it to land.

@mrveiss

mrveiss commented Sep 19, 2026

Copy link
Copy Markdown
Owner Author

Branch issue-16104-batching-heading (final tip 7831564f60) retired, no vehicle inclusion needed. Its one commit (ec668797d7) duplicated the same ATX-heading fix that landed independently via #16129 (superseding this PR, per the #16104 closing comment). Merged current origin/main into the branch to verify: after conflict resolution, scripts/check_pr_issue_batching.py and its test file are byte-identical to origin/main (git diff origin/main HEAD -- <files> empty) — confirmed nothing here is still needed. Deleting the branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant