Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions .github/workflows/pr-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,13 +203,34 @@ jobs:
echo "$auth_status"
scopes_line="$(printf '%s\n' "$auth_status" | grep 'Token scopes:' || true)"
normalized_scopes="$(printf '%s' "$scopes_line" | sed "s/[',]/ /g")"
required_scopes=(repo read:org)
for required_scope in "${required_scopes[@]}"; do
if ! grep -qE "(^|[[:space:]])${required_scope}([[:space:]]|$)" <<< "$normalized_scopes"; then
echo "::error::GH_TOKEN is missing required scope: ${required_scope}"

# Fine-grained PATs don't expose OAuth-style scopes in `gh auth status` —
# the CLI omits the "Token scopes:" line or reports it cannot determine
# scopes. Skip validation in that case (emit a warning only).
# For classic PATs, validate both 'repo' and 'read:org':
# - 'repo' alone is insufficient: `gh pr view --json reviewRequests`
# hard-fails on PRs with team reviewers when 'read:org' is absent.
if [ -z "$normalized_scopes" ] || printf '%s' "$auth_status" | grep -qiE '(cannot determine|fine.grained)'; then
echo "::warning::Token scopes could not be determined (fine-grained PAT). Skipping scope validation."
echo "::warning::Ensure the token has: contents:read and pull_requests:write (fine-grained), or repo + read:org (classic PAT)."
elif grep -qE "(^|[[:space:]])repo([[:space:]]|$)" <<< "$normalized_scopes"; then
# Classic PAT with repo scope — also require read:org for team reviewer support
if ! grep -qE "(^|[[:space:]])read:org([[:space:]]|$)" <<< "$normalized_scopes"; then
echo "::error::GH_TOKEN has 'repo' scope but is missing 'read:org'."
echo "::error::PRs with team-based reviewers will fail at 'gh pr view --json reviewRequests'."
echo "::error::Edit the classic PAT and check the 'read:org' box (no regeneration needed)."
exit 1
fi
done
else
# Classic PAT without full repo scope — verify minimal scopes
for required_scope in contents pull_requests; do
if ! grep -qE "(^|[[:space:]])${required_scope}([[:space:]]|$)" <<< "$normalized_scopes"; then
Comment on lines +226 to +227

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Stop parsing OAuth scopes for fine-grained PATs

For runs that actually use a fine-grained PAT, this validation still fails before any review starts because gh auth status exposes OAuth scopes from the Token scopes: line, not fine-grained repository permissions such as contents or pull_requests; GitHub CLI’s own auth docs describe fine-grained PATs as having inherent resource scoping, and the CLI maintainers note that fine-grained permission information is not readily available for gh to detect. As a result, the recommended token path hits this loop with no matching scopes and exits 1 even when the token has the requested permissions; validate fine-grained tokens with a cheap API probe instead of checking Token scopes: for permission names.

Useful? React with 👍 / 👎.

echo "::error::GH_TOKEN is missing required scope: ${required_scope}"
echo "::error::Token must have either 'repo' + 'read:org' scopes (classic) or 'contents' + 'pull_requests' (fine-grained)"
exit 1
fi
done
fi
Comment thread
don-petry marked this conversation as resolved.

- name: Install review engine CLIs
run: |
Expand Down
15 changes: 14 additions & 1 deletion agents/pr-reviewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,19 @@ tools: ["read", "edit", "search", "execute", "web"]

You are the PR Review Agent for the petry-projects organization.

## Required GitHub token scopes

The agent requires a GitHub personal access token (PAT) with these minimum permissions:

**Repository permissions (fine-grained):**
- `contents:read` — read file contents and diffs for analysis
- `pull_requests:write` — post reviews and comments on PRs

**Optional organizational permissions:**
- `members:read` — read organization members for code owner routing at escalation time

The minimum viable token is a fine-grained PAT with `contents:read` + `pull_requests:write`. Classic PATs (`repo` + `read:org`) are also supported as a fallback — the `repo` scope is not required but is accepted; see bot-setup.md for known limitations of fine-grained PATs in some org configurations.

## Your role

You review pull requests using a cascading tier system that minimizes token spend
Expand All @@ -22,7 +35,7 @@ while maintaining review quality:

| Condition | Action |
|-----------|--------|
| LOW risk, CI passing | Approve and enable auto-merge |
| LOW risk, CI passing | Approve (rebase branch if behind base) |
| MEDIUM risk, CI passing | Approve with detailed findings |
| HIGH risk or CI failing | Escalate to human reviewer |

Expand Down
74 changes: 44 additions & 30 deletions docs/pr-review-agent/pr-review-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,13 @@ and **Copilot**.

4. **Post-review actions** — after the review is posted, the action tier takes
additional actions depending on the decision:
- **If approved:** enables auto-merge (`gh pr merge --auto --squash`),
rebases the branch if behind base, and removes the `needs-human-review`
label. GitHub merges automatically once all required checks pass.
- **If approved:** removes the `needs-human-review` label if present, and
rebases the branch if it's behind base. The PR remains open for manual merge
by the author or maintainers.
- **If escalated + AI delegation enabled:** posts a follow-up comment
with specific fix instructions. An AI agent watches for these comments,
pushes fixes → next cron tick detects new SHA → cascade re-reviews →
approve + auto-merge when clean. This creates an autonomous fix loop.
approve when clean. This creates an autonomous fix loop.
- **If escalated + no delegation (or max cycles reached):** labels
`needs-human-review` and re-requests don-petry as reviewer.
- **Cycle guard:** before running the cascade, `scripts/review-one-pr.sh`
Expand Down Expand Up @@ -129,39 +129,50 @@ authored by `don-petry`; the bot approves them.
**Invite member** → enter `donpetry-bot` → Role: **Member**.
6. Accept the invite from the bot account.

### 2. Create a **classic** PAT for the bot

> [!IMPORTANT]
> **Fine-grained PATs do not work for this workflow.** Use a classic PAT only.
>
> Fine-grained tokens are blocked by org policy gates: even after the org owner
> approves the token request and the bot has Write collaborator access, the
> GraphQL `addPullRequestReview` mutation fails with:
>
> ```
> failed to create review: GraphQL: Resource not accessible by personal access token (addPullRequestReview)
> ```
>
> If you see that error in a workflow run, the secret holds a fine-grained
> token. Replace it with a classic PAT generated as below. The same gate also
> blocks the rulesets bypass that branch protections rely on.
### 2. Create a PAT for the bot (classic or fine-grained)

You can use either a **classic** or **fine-grained** PAT. Classic PATs are the
safe default — see [bot-setup.md](bot-setup.md) for a known failure with
fine-grained tokens in some org configurations (`addPullRequestReview` blocked
at the org policy layer). Fine-grained tokens work when your org permits them
and follow the principle of least privilege.

Comment on lines +132 to +139

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine Grained PAT is working bot the pr-review agent

#### Option A: Fine-grained PAT (recommended)

1. Sign in as the bot account (e.g. `donpetry-bot`) — sign out of `don-petry`
first, or use a private window. The PAT must be created **from the bot's
account**, not yours.
2. Go to **Settings → Developer settings → Personal access tokens →
Tokens (classic)** → **Generate new token (classic)**.
Fine-grained tokens** → **Generate new token**.
3. Settings:
- **Note:** `pr-review-agent`
- **Expiration:** 1 year (set a calendar reminder to rotate)
- **Scopes:** ✅ `repo`, ✅ `workflow`, ✅ `read:org`
- **Token name:** `pr-review-agent`
- **Expiration:** 90 days (set a calendar reminder to rotate — fine-grained PATs do not auto-rotate)
- **Resource owner:** select your organization (e.g. `petry-projects`)
Comment on lines +148 to +150

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's clear enough.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document the single-owner limit for fine-grained PATs

With this fine-grained setup, selecting petry-projects as the resource owner only grants access to repositories owned by that organization, but the workflow still enumerates both the bot account namespace and the target org (scripts/list-prs.sh calls search_namespace "$BOT_USER" before search_namespace "$TARGET_ORG"). In deployments that rely on the bot's personal repos being scanned, following these instructions will make those PRs disappear from the candidate pool; the docs should either require a classic PAT for multi-owner coverage or explicitly state that fine-grained PATs only support the selected owner.

Useful? React with 👍 / 👎.

- **Repository access:** All repositories (or specific repos if preferred)
- **Repository permissions:**
- `contents:read` — read files and diffs for analysis
- `pull_requests:write` — post reviews and comments
- **Organization permissions:**
- `members:read` — (optional) read org members for code owner routing at escalation time

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require members:read for team reviewer escalation

When a repo's CODEOWNERS contains team entries (for example @org/team) and the cascade escalates, scripts/request-codeowners-review.sh passes those entries to gh pr edit --add-reviewer; GitHub CLI team-reviewer operations with fine-grained PATs need the organization Members permission to resolve the team. Marking members:read as optional means users can create the recommended token and still have escalations silently fail to request the team reviewers, leaving only the label fallback.

Useful? React with 👍 / 👎.

4. Generate and copy the token immediately.
5. Sign back in as `don-petry` and store the token in the agent repo's secret
(the secret name is `DON_PETRY_BOT_GH_PAT` in `petry-projects/.github-private`).

After saving, trigger a workflow run and confirm the install step's
`gh auth status` reports the bot's login (not yours) and lists the three
scopes above.
#### Option B: Classic PAT (legacy)

If you prefer classic PATs or need broader scopes:

1. Sign in as the bot account, go to **Settings → Developer settings →
Personal access tokens → Tokens (classic)** → **Generate new token (classic)**.
2. Settings:
- **Note:** `pr-review-agent`
- **Expiration:** 1 year
- **Scopes:** ✅ `repo`, ✅ `workflow`, ✅ `read:org`
3. Generate and store as `DON_PETRY_BOT_GH_PAT` (same as above).

After saving either token type, trigger a workflow run and confirm the
`gh auth status` output reports the bot's login (not yours) and lists the
required scopes.

> **Branch protection / rulesets:** add `donpetry-bot` as an allowed
> approver on each protected repo. In the repo ruleset or branch protection
Expand Down Expand Up @@ -290,9 +301,12 @@ PR comment "@donpetry-bot please review"
1. Copy [`templates/mention-listener.yml`](templates/mention-listener.yml) to
`petry-projects/.github` as `.github/workflows/pr-review-mention.yml`.

2. Add the `DON_PETRY_BOT_GH_PAT` secret to `petry-projects/.github`
(org-level secret or repo secret on `.github`). Use a classic PAT from
`donpetry-bot` with scopes: ✅ `repo`, ✅ `workflow`, ✅ `read:org`
2. Ensure the `GH_PAT_WORKFLOWS` org-level secret is available to
`petry-projects/.github`. This secret is already present in the org; no
additional secret setup is required. The workflow template references
`GH_PAT_WORKFLOWS` throughout — use the same PAT created above in
Comment on lines +304 to +307

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the bot PAT requirement for mention setup

In the mention-trigger setup, this now says no additional secret is required and that the template references GH_PAT_WORKFLOWS throughout, but the checked-in caller workflow still declares and passes DON_PETRY_BOT_GH_PAT as a required org secret for acknowledgement comments (.github/workflows/pr-review-mention.yml lines 22-23 and 41-43). For any repo following these instructions without exposing DON_PETRY_BOT_GH_PAT, the reusable call will receive an empty/missing bot token and the mention trigger cannot post acknowledgements as documented.

Useful? React with 👍 / 👎.

[step 2](#2-create-a-pat-for-the-bot-classic-or-fine-grained) if you need to
rotate or recreate it.

3. Ensure `donpetry-bot` has at least **Read** collaborator access on
`petry-projects/.github-private`.
Expand Down
8 changes: 1 addition & 7 deletions scripts/post-pr-review.sh
Original file line number Diff line number Diff line change
Expand Up @@ -248,16 +248,10 @@ if [ "$DECISION" = "approve" ]; then
fi
fi

# Enable auto-merge (skip if branch is still behind — GitHub would block it anyway)
if [ "$MERGE_STATE" != "BEHIND" ]; then
echo "Enabling auto-merge..."
gh pr merge "$PR_URL" --auto --squash 2>/dev/null || true
fi

# Clean up label
gh pr edit "$PR_URL" --remove-label needs-human-review 2>/dev/null || true

echo "Review posted and auto-merge enabled"
echo "Review posted"

elif [ "$DECISION" = "escalate" ]; then
# Check if AI delegation should be used
Expand Down
Loading