feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues - #1343
Conversation
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 30 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR adds net-zero diff protection to dev-lead fix flows, updates prompts to ignore neutral reviews, and repins the PR review mention workflow to its stable channel while adjusting drift-check handling. ChangesNo-op fix protection
Workflow repin
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant FixIntent
participant commit_and_push
participant GitHub
FixIntent->>commit_and_push: process fix intent
commit_and_push->>commit_and_push: compare base-to-head diff
commit_and_push->>GitHub: flag net-zero PR and disable auto-merge
commit_and_push-->>FixIntent: return status 3
FixIntent-->>GitHub: skip push and terminal cleanup
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #1343 |
There was a problem hiding this comment.
Code Review
This pull request implements a no-op guard mechanism to prevent pushing self-cancelling fixes that result in a net-zero diff against the base branch, which would otherwise auto-close compliance issues without resolving them. It updates prompt guidelines, adds a net-zero check and flagging logic in the script, and introduces corresponding unit tests. The review feedback suggests improving the newly added BATS tests by using $BATS_TEST_TMPDIR for isolated test execution and ensuring that negative assertions on grep check for an exact exit status of 1 to avoid false passes on unexpected errors.
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
There was a problem hiding this comment.
Pull request overview
Implements the #1340 mitigation in the dev-lead “fix” loop to prevent self-cancelling fix passes (where the bot reverts its own fix and produces a net-zero PR diff that could still auto-close Closes #N compliance issues). This adds both behavioral guardrails in the fix workflow and prompt-level guidance to avoid treating neutral Copilot “overview” content as actionable review feedback.
Changes:
- Add a no-op guard in
dev-lead-fix-reviews.shthat detects when the PR’s net diff vs base is empty after a fix pass, then flags the PR, disables auto-merge, and refuses to push. - Update fix-reviews / fix-bot-comment control flow to handle the new “no-op guard aborted push” outcome without posting misleading terminal markers or re-enabling auto-merge.
- Extend dev-lead prompts and unit tests to cover #1340 behaviors (no-op guard + “COMMENTED overview is neutral” guidance).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| tests/dev-lead/unit/test_fix_reviews.bats | Adds unit coverage for the new net-zero no-op guard and prompt guidance assertions. |
| scripts/dev-lead-fix-reviews.sh | Implements the net-zero diff guard + PR flagging/escalation behavior and integrates new return code handling. |
| prompts/dev-lead/fix-reviews.md | Adds explicit instruction that COMMENTED/overview reviews are neutral and must not trigger reverts. |
| prompts/dev-lead/fix-bot-comment.md | Adds explicit instruction to ignore neutral overview/summary bot comments unless they contain actionable findings. |
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
…channel and updated guidance comments
Dev-Lead — fix-bot-comment (applied)Changes committed and pushed. |
Dev-Lead — fix-bot-comment (applied)Changes committed and pushed. |
Dev-Lead — fix-bot-comment (applied)Changes committed and pushed. |
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
Auto-dismissed (#617): coderabbitai[bot] CHANGES_REQUESTED on a superseded commit. The bot re-reviews the new head automatically — a valid concern will return as a fresh review.
|
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: 903647dda751fb3e109e9687feceec38f30ee9a3
Review mode: triage-approved (single reviewer)
Summary
Implements issue #1340's two proposed fixes for the dev-lead fix-bot self-cancelling loop: (1) explicit prompt guidance in fix-reviews.md and fix-bot-comment.md that COMMENTED/overview reviews are neutral and must never trigger reverting the PR's own changes, and (2) a no-op guard in scripts/dev-lead-fix-reviews.sh (pr_nets_to_zero + flag_noop_pr) that refuses to push when the base…head diff nets to zero, posts a deduped human-attention comment, adds the needs-human-review label, disables auto-merge, and suppresses the EXIT-trap auto-merge restore. Callers propagate the new return code 3 so no false 'applied' marker is posted and threads are not auto-resolved. Six new bats tests cover both intents, the non-net-zero happy path, auto-merge suppression, and prompt content. Two bundled maintenance changes: pr-review-mention.yml moves from @pr-review-mention/v2-next to the @pr-review-mention/stable channel (sanctioned mutable first-party ref per AGENTS.md 'Release channel tags & the mutable-ref exception'; caller-stub-freeze, stub-structure, and template-drift checks all pass), and template_stub_drift.sh normalizes trailing newlines symmetrically when hashing the template side — a narrow fix that does not mask genuine content drift, replacing an earlier broader allowlist approach that CodeRabbit correctly flagged (thread resolved).
Linked issue analysis
Issue #1340 (open): fix-bot misreads neutral Copilot overview comments as change-requests, reverts its own fix, and the resulting net-zero PR would falsely auto-close compliance issues. Both proposed mitigations are substantively implemented: prompt-level 'never act on COMMENTED/overview' guidance and the post-commit net-zero push guard. The guard deliberately fails open (warns and proceeds) when the base is unverifiable — shallow-clone deepening, missing merge-base, and diff-failure cases are each handled explicitly, so a broken environment never blocks a legitimate push. Scope matches the issue; PR correctly carries Closes #1340.
Findings
No blocking findings.
- Correctness: return-code-3 plumbing is sound — cp_rc=1 (no changes) still flows to the existing no-changes path; cp_rc=3 skips thread resolution, terminal markers, and try_enable_auto_merge; _AM_NEEDS_RESTORE=0 prevents the EXIT trap from re-enabling auto-merge. Guard is scoped to fix-reviews/fix-bot-comment intents only.
- Security: no injection vectors (comment bodies built from internal vars via gh CLI args); no permission or secrets-mapping changes in the workflow stub (secrets lines are unchanged context); NOSONAR on the channel ref is documented and consistent with AGENTS.md's audited mutable-ref exception.
- Minor (non-blocking): _template_drift_committed_sha classifies a genuinely empty or >1MB template file (contents API omits inline content) as MISSING rather than comparing; acceptable for small workflow stubs.
- Secret scan: run_secret_scanning MCP tool not available in this environment — noted, not fabricated; gitleaks CI check passed.
CI status
All required checks green at 903647d: shellcheck, bats, unit-tests, CodeQL, SonarCloud, Agent Security Scan, AgentShield, gitleaks, actionlint, template-drift, caller-stub-freeze, stub-structure, validate-caller-inputs, caller-permissions, and both later 'review / review' runs SUCCESS. One earlier 'review / review' run shows CANCELLED — superseded by a subsequent successful run of the same check at the same SHA (concurrency cancellation), not a failure. All 12 review threads (gemini, copilot, coderabbit) are resolved, including CodeRabbit's Major drift-allowlist finding, which the final revision replaced with the narrower newline-normalization fix.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
Dev-Lead — waiting on PR blockers (intent: fix-reviews)PR: #1343 |
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>
… neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues (#1343) * feat: implement issue #1340 — dev-lead fix-bot reverts its own fix on neutral Copilot 'overview' comments → net no-op PRs that falsely close compliance issues * fix(reviews): address review comments [skip ci-relay] * fix: align pr-review-mention.yml with updated standards — use stable channel and updated guidance comments * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: don-petry <{}+don-petry@users.noreply.github.com>



Closes #1340
Implemented by dev-lead agent. Please review.
Summary by CodeRabbit
Bug Fixes
Maintenance