fix(dev-lead): resolve review threads after fixing + notify CodeRabbit on CHANGES_REQUESTED - #285
Conversation
Three related gaps closed:
1. Thread resolution after fixing
- fix-reviews, fix-bot-comment, and human-pr prompts now instruct the
agent to resolve each thread immediately after addressing it (via the
resolveReviewThread GraphQL mutation) and to resolve any thread marked
isOutdated without a code change. This gives reviewers (human and bot)
a clean slate to re-review rather than leaving stale CHANGES_REQUESTED
state indefinitely.
2. isOutdated + id fields in GraphQL queries
- Both reviewThreads queries in dev-lead-fix-reviews.sh now fetch id and
isOutdated so the agent has the data it needs to resolve threads.
3. CodeRabbit CHANGES_REQUESTED handshake
- Added notify_coderabbit_resolve() which posts @coderabbitai resolve
after a successful commit+push whenever coderabbitai[bot] has an
outstanding CHANGES_REQUESTED review. This triggers CodeRabbit to mark
all its threads resolved and post APPROVED (when request_changes_workflow
is enabled), clearing the stale review state that previously persisted
even after all findings were addressed in code.
The old "Do not resolve the threads yourself" instruction was wrong: it
was intended to mean "don't use GitHub's thread-resolve API yourself" but
inadvertently blocked the correct resolution workflow for all reviewers.
Closes #281 (partial — CodeRabbit resolution gap)
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe PR updates the dev-lead agent workflow to automatically resolve review threads. Three agent prompt templates are revised to instruct agents to resolve fixed and outdated threads using GraphQL mutations, and the bash script is updated with a new notification function and GraphQL query changes to fetch thread IDs and trigger bot resolution comments after successful edits. ChangesThread Resolution Workflow
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism for the dev-lead agent to explicitly resolve review threads. It updates prompt documentation for bot and human reviews with instructions and GraphQL mutations for thread resolution. Additionally, the dev-lead-fix-reviews.sh script was updated to fetch thread IDs and include a new function to notify CodeRabbit to resolve its threads when changes are pushed. Feedback was provided to improve the reliability of the CodeRabbit status check by implementing pagination and targeting only the most recent review state.
| has_cr_changes_requested=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/reviews" \ | ||
| --jq '[.[] | select(.user.login == "coderabbitai[bot]" and .state == "CHANGES_REQUESTED")] | length' \ | ||
| 2>/dev/null || echo "0") |
There was a problem hiding this comment.
The current check for CodeRabbit's review status has two issues:
- It doesn't use
--paginate, which means it only inspects the first 30 reviews. On long-lived PRs, the most recent (and relevant) reviews might be missed since GitHub returns them in chronological order. - It checks if any review from CodeRabbit is in
CHANGES_REQUESTEDstate. If a bot previously requested changes but has since approved the PR in a subsequent review, this logic will still trigger unnecessarily.
It's better to use --paginate and check only the state of the latest review from the bot.
| has_cr_changes_requested=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/reviews" \ | |
| --jq '[.[] | select(.user.login == "coderabbitai[bot]" and .state == "CHANGES_REQUESTED")] | length' \ | |
| 2>/dev/null || echo "0") | |
| has_cr_changes_requested=$(gh api --paginate "repos/${REPO}/pulls/${PR_NUMBER}/reviews" \ | |
| --jq '[.[] | select(.user.login == "coderabbitai[bot]")] | last | if .state == "CHANGES_REQUESTED" then 1 else 0 end' \ | |
| 2>/dev/null || echo "0") |
There was a problem hiding this comment.
Pull request overview
This PR updates the dev-lead automation to actively resolve review threads it fixes (including isOutdated threads) and to notify CodeRabbit to clear lingering CHANGES_REQUESTED states after a successful push.
Changes:
- Add
notify_coderabbit_resolve()to post@coderabbitai resolvewhen CodeRabbit has requested changes, and invoke it after successful pushes for relevant intents. - Extend
reviewThreadsGraphQL queries to includeidandisOutdatedso threads can be resolved viaresolveReviewThread. - Update dev-lead prompt templates to instruct resolving threads after fixes (and resolving outdated threads without code changes).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| scripts/dev-lead-fix-reviews.sh | Adds CodeRabbit resolve notification logic and enriches thread queries with id/isOutdated. |
| prompts/dev-lead/fix-reviews.md | Updates guidance to resolve fixed/outdated threads via GraphQL mutation. |
| prompts/dev-lead/fix-bot-comment.md | Adds bot-thread resolution instructions and a command snippet to list/resolve threads. |
| prompts/dev-lead/human-pr.md | Adds human-thread resolution instructions and updated summary format. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| echo "[dry-run] would check for coderabbitai CHANGES_REQUESTED and post @coderabbitai resolve" | ||
| return 0 | ||
| fi | ||
| local has_cr_changes_requested | ||
| has_cr_changes_requested=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/reviews" \ | ||
| --jq '[.[] | select(.user.login == "coderabbitai[bot]" and .state == "CHANGES_REQUESTED")] | length' \ | ||
| 2>/dev/null || echo "0") | ||
| if [ "${has_cr_changes_requested:-0}" -gt 0 ]; then | ||
| echo "::notice::coderabbitai has CHANGES_REQUESTED — posting @coderabbitai resolve" |
| --jq '.data.repository.pullRequest.reviewThreads.nodes | ||
| | map(select(.isResolved == false | ||
| and (.comments.nodes[0].author.login == "${ACTOR}" or .isOutdated == true)))' |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@prompts/dev-lead/fix-bot-comment.md`:
- Around line 46-48: The jq filter currently allows matching outdated threads
from any reviewer; update the filter expression used in the shell snippet (the
--jq string) so that outdated threads are only selected when authored by this
bot: require (.comments.nodes[0].author.login == "${ACTOR}") to be true for both
unresolved OR isOutdated cases (i.e., select threads where author == ACTOR and
(isResolved == false or isOutdated == true)); alternatively, if the intended
behavior is to resolve every outdated thread regardless of author, update the
prose at line 59 to explicitly state "Resolve every thread you fixed from this
bot and every thread marked isOutdated: true (regardless of author)" so the
current filter is knowingly correct.
In `@scripts/dev-lead-fix-reviews.sh`:
- Around line 77-94: The notify_coderabbit_resolve function currently posts the
literal "`@coderabbitai` resolve" which force-resolves all CodeRabbit threads;
change it so it never indiscriminately posts that command—either remove the gh
pr comment call entirely or gate it behind an explicit opt-in (e.g., an env var
like ENABLE_CODERABBIT_AUTO_RESOLVE) and only run when that opt-in is true;
alternatively implement targeted resolution logic (use the GH API to inspect
review threads and only post resolves for specific thread IDs that are verified
as fixed/outdated) instead of posting the global "`@coderabbitai` resolve" string;
update references in notify_coderabbit_resolve and the has_cr_changes_requested
check accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 27050f55-dbc7-45ff-a622-705546b1fe65
📒 Files selected for processing (4)
prompts/dev-lead/fix-bot-comment.mdprompts/dev-lead/fix-reviews.mdprompts/dev-lead/human-pr.mdscripts/dev-lead-fix-reviews.sh
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 29bbf558e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| has_cr_changes_requested=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/reviews" \ | ||
| --jq '[.[] | select(.user.login == "coderabbitai[bot]" and .state == "CHANGES_REQUESTED")] | length' \ | ||
| 2>/dev/null || echo "0") |
There was a problem hiding this comment.
In PRs where CodeRabbit requested changes earlier and later approved after fixes, this query still counts the old historical CHANGES_REQUESTED review, so any subsequent successful fix-reviews, fix-bot-comment, or human-pr run will post @coderabbitai resolve even when CodeRabbit has no outstanding request. This repo already treats stale reviews as historical by checking the review's commit SHA in scripts/review-one-pr.sh, so this should similarly restrict to the latest CodeRabbit review or at least a request on the current head SHA before sending the resolve command.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Skipped: comment is from another GitHub bot.
| | map(select(.isResolved == false | ||
| and (.comments.nodes[0].author.login == "${ACTOR}" or .isOutdated == true)))' |
There was a problem hiding this comment.
For a bot-comment run, this filter includes every unresolved outdated thread on the PR, even when the thread was opened by a human or a different bot, because the author check is ORed with isOutdated. In a PR with an unrelated outdated human review thread, the agent is explicitly instructed to resolve it while handling ${ACTOR}'s comment, which violates the surrounding “from this bot” scope and can hide feedback that the bot-comment workflow should leave alone.
Useful? React with 👍 / 👎.
| has_cr_changes_requested=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/reviews" \ | ||
| --jq '[.[] | select(.user.login == "coderabbitai[bot]" and .state == "CHANGES_REQUESTED")] | length' \ | ||
| 2>/dev/null || echo "0") |
There was a problem hiding this comment.
On PRs with more than 30 reviews, this REST call only inspects the first page because it does not pass --paginate or a larger per_page; GitHub's List reviews endpoint defaults per_page to 30 (see https://docs.github.com/rest/pulls/reviews). If CodeRabbit's current CHANGES_REQUESTED review is on a later page, notify_coderabbit_resolve will skip posting @coderabbitai resolve, leaving the outstanding CodeRabbit state uncleared on the busy PRs this helper is meant to fix.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Skipped: comment is from another GitHub bot.
| mutation($id: ID!) { | ||
| resolveReviewThread(input: {threadId: $id}) { | ||
| thread { isResolved } | ||
| } | ||
| }' -F id="<threadId>" |
There was a problem hiding this comment.
These prompt templates are run through envsubst in build_and_run before the agent sees them, so the new GraphQL variables are expanded as shell variables and disappear because $id is not exported. The generated prompt contains invalid GraphQL like mutation(: ID!) and threadId:, so agents following this instruction cannot resolve any review thread; the same envsubst stripping affects the other added GraphQL snippets that use $owner, $repo, $pr, and $id.
Useful? React with 👍 / 👎.
Per Gemini, Copilot, and CodeRabbit review of #285: - notify_coderabbit_resolve: use --paginate so all reviews are seen on long-lived PRs; derive current state from the latest CodeRabbit review (tail -1 on chronological .state output) rather than any historical one, avoiding noisy re-posts after a prior approval or dismissal - fix-bot-comment.md: pass ACTOR as a jq --arg so the variable expands at runtime instead of being compared as the literal string "${ACTOR}"; scope isOutdated resolution to threads from this bot only (not any reviewer) to avoid resolving another reviewer's still-valid threads
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 061c35bff4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
| } | ||
| }' -F owner="${REPO%%/*}" -F repo="${REPO##*/}" -F pr=${PR_NUMBER} \ | ||
| --jq --arg actor "${ACTOR}" \ |
There was a problem hiding this comment.
In fix-bot-comment runs where the agent follows this helper to find threads for ${ACTOR}, this command will fail before returning any thread IDs: the official gh api syntax exposes --jq <expression> only, so --arg actor ... is parsed as an unsupported GitHub CLI flag rather than a jq option. As a result, bot-comment fixes cannot discover or resolve the bot's open/outdated review threads; pass the actor via the jq expression/env or pipe to a separate jq --arg actor ... invocation instead.
Useful? React with 👍 / 👎.
| pullRequest(number:$pr) { | ||
| reviewThreads(first:50) { | ||
| nodes { isResolved line path comments(first:5) { nodes { body author { login } } } } | ||
| nodes { id isResolved isOutdated line path comments(first:5) { nodes { body author { login } } } } |
There was a problem hiding this comment.
When a human review triggers human-pr while bot review threads are also unresolved, this query now hands the agent resolvable id values for every unresolved thread on the PR, not just human threads. The updated human-pr prompt then instructs the agent to resolve every fixed or outdated thread in OPEN_THREADS_JSON, so a human-only response can resolve CodeRabbit/Copilot/Sonar threads in the first page and hide bot feedback that this intent was not scoped to handle.
Useful? React with 👍 / 👎.
| pullRequest(number:$pr) { | ||
| reviewThreads(first:50) { | ||
| nodes { isResolved line path comments(first:5) { nodes { body author { login } } } } | ||
| nodes { id isResolved isOutdated line path comments(first:5) { nodes { body author { login } } } } |
There was a problem hiding this comment.
When a trusted bot review triggers fix-reviews while unresolved human review threads are also present, this query now includes resolvable IDs for every unresolved thread on the PR. Because the new prompt tells the agent to resolve every fixed or outdated thread from OPEN_THREADS_JSON, a bot-review run can mark human feedback resolved even though the intent was only routed from a bot review event.
Useful? React with 👍 / 👎.
| - Do not resolve the threads yourself — they will be resolved automatically when the conversation is updated | ||
| ### Resolving a thread | ||
|
|
||
| After fixing (or confirming outdated), resolve it immediately: |
There was a problem hiding this comment.
In runs where the agent fixes code but commit_and_push later fails because of auth, branch permissions, or a transient push error, this instruction has already resolved the review thread before any fix reaches the PR branch. The workflow then exits and the runner's local changes are lost, leaving reviewer feedback hidden as resolved even though the PR still contains the original issue; resolution should happen only after the script has successfully pushed the fix.
Useful? React with 👍 / 👎.
…by author Per ChatGPT Codex P1/P2 findings on #285: - build_and_run: scope envsubst to variables declared in the prompt's <!-- VARIABLES: --> header. Previously, envsubst replaced all $VAR patterns including GraphQL variables ($id, $owner, $repo, $pr) in code examples, silently stripping them before the agent ever saw the prompt. Now only the declared template variables are substituted. - fix-bot-comment.md: replace --jq --arg (unsupported by gh api graphql) with a pipe to jq --arg, which correctly passes the actor login as a jq variable for the thread-lookup query. - All three prompts (fix-reviews, fix-bot-comment, human-pr): rewrite the resolveReviewThread mutation example to use an inline string literal (THREAD_NODE_ID placeholder) instead of a GraphQL $id variable, making the pattern safe even without the envsubst scoping fix and clearer for the agent to follow. - All three prompts: add constraint to only resolve threads from the reviewer who triggered this run — do not resolve threads belonging to other reviewers (human or bot) to avoid accidentally clearing still- valid concerns from a different reviewer.
|
Problem
The dev-lead prompts said "Do not resolve the threads yourself" — this was wrong in two ways:
Additionally, the script never posted `@coderabbitai resolve` after a successful push, so CodeRabbit's `request_changes_workflow` CHANGES_REQUESTED review never cleared to APPROVED even when all findings were addressed.
Changes
`scripts/dev-lead-fix-reviews.sh`
`prompts/dev-lead/fix-reviews.md`
`prompts/dev-lead/fix-bot-comment.md`
`prompts/dev-lead/human-pr.md`
Correct lifecycle after this fix
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation