feat: implement issue #1105 — [Phase 2] Rewire the security-audit + single-review tier prompts to consume pre-fed context - #1163
Conversation
…ingle-review tier prompts to consume pre-fed context
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 51 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughSecurity-audit and single-review prompts now use SHA-bound pre-fetched metadata and diffs when available, while retaining live-command fallbacks and dynamic context gathering. Cascade-action documents synthesis-only behavior. New Bats coverage validates the prompt rewiring and runs in lint CI. ChangesPre-fed context prompt flow
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request integrates pre-fed PR context (metadata and diff files) into the security audit and single review prompts, allowing them to bypass live GitHub CLI calls when pre-fetched files are available and fresh. It also adds a comprehensive BATS test suite to verify these changes. The reviewer recommended adding a file existence check in the test helper function _prefed_section to prevent awk from hanging on standard input if the file argument is missing.
There was a problem hiding this comment.
Pull request overview
This PR implements issue #1105 (epic #1101, Phase 2/Story 5) by updating the tier-3 security-audit and single-review prompts to consume pre-fetched PR metadata/diff files when available, while retaining the original gh pr view / gh pr diff behavior as the fallback. It also documents that the cascade action step does not fetch PR context and adds a structural Bats guard to prevent regressions.
Changes:
- Add a “Pre-fed PR context” section to
prompts/security-audit.mdandprompts/single-review.md, instructing the agent to read$PR_CONTEXT_METADATA_FILE/$PR_CONTEXT_DIFF_FILEwhen present/fresh (matching$PR_HEAD_SHA), otherwise fall back togh pr view/gh pr diff. - Document
prompts/cascade-action.mdas explicitly out-of-scope (it synthesizes$FINAL_RESULTand does not fetch diff/metadata). - Add
tests/test_prefetch_prompt_rewire.batsand wire it into thelint.ymlBats suite.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_prefetch_prompt_rewire.bats |
Adds an offline structural guard ensuring the new “Pre-fed PR context” sections exist and key behaviors remain present. |
prompts/single-review.md |
Instructs single-review to consume pre-fed metadata/diff when present and fresh; otherwise preserve existing fetch steps. |
prompts/security-audit.md |
Same pre-fed-context rewire for the audit tier while keeping dynamic gh api/MCP/LSP guidance intact. |
prompts/cascade-action.md |
Documents that cascade-action does not fetch PR context and is out of scope for the rewire. |
.github/workflows/lint.yml |
Adds the new Bats test to the existing lint test list. |
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_prefetch_prompt_rewire.bats`:
- Around line 49-61: Add behavioral tests around the context-selection predicate
used by the pre-fed audit section, rather than only checking its prose. Cover
missing pre-fed files falling back to gh, mismatched PR_HEAD_SHA falling back,
and header-only diffs being treated as empty or invalid context; use the
existing test helpers and fixtures around _prefed_section and the audit flow to
assert each outcome.
- Around line 118-121: The test for cascade-action in
“tests/test_prefetch_prompt_rewire.bats” only verifies documentation and must
also verify implementation. Update the test named “cascade-action prompt is
documented as not fetching the diff (out of scope)” to add negative assertions
confirming $ACTION contains neither “gh pr view” nor “gh pr diff”, while
retaining the existing documentation assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 83c61fa2-5cd2-4ad3-ad22-ff4b4eebc7a2
📒 Files selected for processing (5)
.github/workflows/lint.ymlprompts/cascade-action.mdprompts/security-audit.mdprompts/single-review.mdtests/test_prefetch_prompt_rewire.bats
Superseded by automated re-review at
|
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
Superseded by automated re-review at
|
|
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #1163 |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 88e09e6c8d9aeefbde6c613b37ba777c5e2f93e8
Review mode: triage-approved (single reviewer)
Summary
Re-review (cycle 2/3) after cycle-1 fix-request. Prompt-only rewire for epic #1101 Story 5: adds a 'Pre-fed PR context' section to prompts/security-audit.md and prompts/single-review.md so they consume $PR_CONTEXT_METADATA_FILE / $PR_CONTEXT_DIFF_FILE when present and SHA-fresh, with explicit byte-identical fallback to gh pr view / gh pr diff; documents prompts/cascade-action.md as out of scope; adds a 14-case structural bats guard registered in lint.yml. Both cycle-1 blocking-state findings are now resolved: the delta commit (536757e..88e09e6, +3 lines) adds the requested negative assertions that cascade-action.md contains no 'gh pr view' / 'gh pr diff' commands, all five review threads are resolved, and CodeRabbit dismissed its CHANGES_REQUESTED review and APPROVED. No scripts or workflows beyond the lint.yml test registration are touched; no security-sensitive surface.
Linked issue analysis
Closes #1105. AC1 (both prompts consume pre-fed metadata/diff when vars set, files exist, and pr_head_sha stamp matches): met. AC2 (byte-identical fallback when flag off / files absent / stamp stale): met via the explicit Otherwise clause. AC3 (dynamic gh api / MCP / LSP steps retained — audit standards fetch, LSP verification, incremental compare, secret scan, linked-issue fetch): met and guarded by bats assertions. AC4 (cascade-action.md confirmed out of scope): met via doc note plus negative-assertion test added this cycle. AC5 (live smoke run with flag on): not evidenced in-PR; the test header documents the bats checks as offline structural guards and the prefetch flag is default-off, so this is a non-blocking follow-up — flagging for the epic's Story 6 A/B eval to confirm.
Findings
- [RESOLVED] Cycle-1 finding: cascade-action out-of-scope test passed on prose alone. Fixed at 88e09e6 — the test now also asserts cascade-action.md contains no 'gh pr view' / 'gh pr diff' commands (negative greps).
- [RESOLVED] Cycle-1 blocking state: CodeRabbit CHANGES_REQUESTED standing against the reviewed SHA. CodeRabbit has since dismissed that review and APPROVED (13:12:32Z); all 5 review threads (gemini, copilot x2, coderabbit x2) are resolved; reviewDecision is no longer CHANGES_REQUESTED.
- [NOTE] AC5 smoke run with the prefetch flag on remains unevidenced; acceptable given the flag is default-off and Story 6's eval covers systemic regressions.
- [NOTE] run_secret_scanning MCP tool unavailable in this session; gitleaks CI check is green and the diff (prompt prose + bats tests only) introduces no secrets or security anti-patterns.
No new issues introduced by the delta commit.
CI status
All checks green at 88e09e6: Lint, shellcheck/ShellCheck, bats, unit-tests, CodeQL (actions+python), SonarCloud (Quality Gate passed), Secret scan (gitleaks), agent-shield, Agent Security Scan, holdout-guard, template-drift, gh-aw-compile, validate-agent-profiles, guard, CodeRabbit. Cancelled dev-lead dispatch / review entries are superseded relay runs with later SUCCESS for the same checks; dependency-audit jobs skipped (no matching ecosystems). mergeStateStatus BLOCKED only pending this required review.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
Closes #1105
Implemented by dev-lead agent. Please review.
Summary by CodeRabbit
New Features
Tests