feat: implement issue #408 — Compliance: ruleset-drift-pr-quality-dismiss_stale_reviews_on_push - #434
Conversation
…miss_stale_reviews_on_push
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 48 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughDry-run mode now avoids live ruleset API calls. A Bash test validates the generated ruleset payload and runs in CI. Workspace dependency overrides require newer minimum versions. ChangesRuleset dry-run
Workspace dependency overrides
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates scripts/apply-rulesets.sh to support an offline dry-run mode by skipping live GitHub API calls when DRY_RUN is enabled. It also introduces a new test script, tests/scripts/apply-rulesets.test.sh, to validate the codified ruleset configurations. Feedback is provided to optimize the test assertions using Bash here-strings and grep -F, and to simplify the script existence check since executable permissions are not strictly required when running via bash.
There was a problem hiding this comment.
Pull request overview
This PR addresses compliance issue #408 by adding a regression test to ensure the codified pr-quality ruleset includes dismiss_stale_reviews_on_push: true, and by adjusting the ruleset application script so --dry-run does not perform live GitHub API fetches.
Changes:
- Add a shell test that runs
scripts/apply-rulesets.sh --dry-run --forceand asserts the emitted ruleset JSON includes the compliance-critical fields. - Update
scripts/apply-rulesets.shto skip fetching existing rulesets from GitHub when--dry-runis enabled (treating existing rulesets as[]).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/scripts/apply-rulesets.test.sh | Adds a dry-run-based regression test asserting the pr-quality payload includes dismiss_stale_reviews_on_push: true and other key parameters. |
| scripts/apply-rulesets.sh | Makes --dry-run avoid the live gh api repos/.../rulesets fetch to allow offline-ish dry runs and deterministic output. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
🤖 CodeAnt AI — Review Status
|
codeant-ai
Bot
added
the
size:M
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
codeant-ai
Bot
added
size:L
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Line 169: Update the actions/checkout step in the test job to disable
credential persistence by configuring its persist-credentials option as false.
Keep the existing pinned checkout revision and all other workflow behavior
unchanged.
In `@tests/scripts/apply-rulesets.test.sh`:
- Around line 59-60: Update the _dismiss extraction and assertion to validate
that .parameters.dismiss_stale_reviews_on_push is a JSON boolean before
accepting true; perform the type check inside jq and print only the validated
boolean result, preserving false as the default for missing values.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 2fad0e22-d4fc-403c-81e1-d8766def4eba
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (3)
.github/workflows/ci.ymlpnpm-workspace.yamltests/scripts/apply-rulesets.test.sh
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
|
Dev-Lead — waiting on PR blockers (intent: fix-reviews)PR: #434 |
Dev-Lead — fix-reviews (no-changes)Agent reasoning |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: 77c334af237d813df0767e33b318c73df3b7b7b9
Review mode: triage-approved (single reviewer)
Summary
Compliance fix for issue #408: makes apply-rulesets.sh --dry-run fully offline/deterministic (skips the live rulesets fetch), adds a shell test asserting the codified pr-quality ruleset sets dismiss_stale_reviews_on_push=true plus the other standard PR-protection parameters, wires the test into CI as a new Shell Tests job, and bumps override floors for postcss/brace-expansion (with lockfile updates for postcss, brace-expansion, nanoid).
Linked issue analysis
Issue #408 (ruleset-drift-pr-quality-dismiss_stale_reviews_on_push) reports the pr-quality ruleset drifted from the codified standard (dismiss_stale_reviews_on_push expected true). This PR verifies the codified payload emits the correct value and adds a CI guard so a regression cannot land silently; the test also pins the remaining standard pull_request parameters (1 approving review, code-owner review, last-push approval, thread resolution) scoped to the pr-quality ruleset object so a match in another ruleset cannot produce a false pass. Substantively addresses the finding — actual convergence remains a run of scripts/apply-rulesets.sh with admin scope, per the issue's remediation.
Findings
- Triage assessment confirmed: no issues that would block approval.
- Workflow change is clean: new Shell Tests job uses actions/checkout pinned to 9c091bb2 (verified via GitHub API to be the v7.0.0 tag), persist-credentials: false, no secrets, no untrusted-input interpolation.
- Test hygiene is good: a mock gh shim is first on PATH and hard-fails on any 'gh api' invocation, so a dry-run regression cannot silently make a live API call; assertions use grep -qF with fixed strings and jq scoped to the pr-quality object.
- Dependency changes are patch-level floor bumps of existing overrides (postcss >=8.5.18, brace-expansion >=5.0.8) with matching lockfile resolution (postcss 8.5.25, brace-expansion 5.0.9, nanoid 3.3.16); pnpm audit is green.
- Minor (non-blocking): the test filters info lines with grep -v '^[' before jq -s, which assumes payload JSON never starts a line with '['; acceptable given the script's current output format.
- Advisory bot feedback (gemini here-string/grep -F suggestion) is already applied in the current head. SonarCloud quality gate passed.
- Secret scan: the run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check passed on the head SHA and no secrets are present in the diff.
CI status
All code CI checks on head 77c334a are green: TypeScript, Go, Shell Tests, CodeQL (3 analyses), SonarCloud, SonarCloud Code Analysis, Secret scan (gitleaks), agent-shield, dependency-audit (pnpm audit, govulncheck), Graphite AI Reviews. Cancelled/skipped entries (dev-lead dispatch/ci-relay, review/review) are superseded agent-orchestration runs, each with a later success/skipped run of the same workflow; they are not code CI.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
User description
Closes #408
Implemented by dev-lead agent. Please review.
Summary by CodeRabbit
New Features
Tests
CodeAnt-AI Description
Make ruleset dry runs offline and verify pull request review protections
What Changed
pr-qualitydismisses stale reviews after pushes and requires the standard approval protectionsImpact
✅ Offline ruleset validation✅ Enforced stale-review dismissal✅ Automated pull request protection checks💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.