PR Review Agent — failures detected 2026-05-20

[github-actions]

PR Review Agent — Health Report

Generated: 2026-05-20 | Workflow: pr-review.yml | Repo: petry-projects/.github-private

---

1. Executive Summary

Status: DEGRADED
Period: 2026-05-20 (last 24 hours)
Result: 3 of 100 runs failed (3%) | 41 cancelled (41%) | 8 skipped (8%) | 48 succeeded (48%)

Key findings:

• All 3 failures share a single root cause: GH_TOKEN is empty when workflow is triggered by Dependabot, causing gh auth status to exit 1 and kill the job immediately
• Failures affect Dependabot PRs only (Secret source: Dependabot logged in all three) — secrets.DON_PETRY_BOT_GH_PAT and CLAUDE_CODE_OAUTH_TOKEN are inaccessible in Dependabot context
• 41% cancellation rate in a tight 05:05–05:10 window suggests a Dependabot-triggered thundering herd: many concurrent runs queued, with most cancelled before execution (likely via PR merge/close events draining the queue)

Action required: Add DON_PETRY_BOT_GH_PAT and CLAUDE_CODE_OAUTH_TOKEN to the repo's Dependabot secrets store, or add a job-level guard to skip Dependabot-triggered runs.

---

2. Failure Breakdown

Failure Category	Affected Runs	Example Error Message
Auth / missing secret (Dependabot context)	#1187, #1146, #1144 (3 total)	You are not logged into any GitHub hosts. To log in, run: gh auth login
Cancellation (concurrency queue drain)	#1199, #1200, #1201 … (41 total)	(no error log — cancelled before runner reached steps)
Other / unknown	0	—

---

3. Error Patterns

Pattern 1 — gh auth status fails immediately (all 3 failures)

Exact error:

You are not logged into any GitHub hosts. To log in, run: gh auth login
##[error]Process completed with exit code 1.

Step: "Install review engine CLIs" — first executable line: gh auth status

Root cause: The job-level GH_TOKEN env var is sourced from secrets.DON_PETRY_BOT_GH_PAT:

GH_TOKEN: ${{ secrets.DON_PETRY_BOT_GH_PAT }}

When GitHub Actions runs a workflow in response to a Dependabot event, the runner is provisioned with Secret source: Dependabot. In this mode, only secrets explicitly stored in the Dependabot secrets namespace (Settings → Secrets and variables → Dependabot) are injected. Standard repository/org secrets — including DON_PETRY_BOT_GH_PAT, CLAUDE_CODE_OAUTH_TOKEN, COPILOT_GITHUB_TOKEN, and GOOGLE_API_KEY — are all redacted to empty strings. gh finds no GH_TOKEN and no GITHUB_TOKEN login; gh auth status fails; set -euo pipefail propagates the exit code and the job dies before touching a single PR.

Confirmed by log evidence in all 3 runs:

  GH_TOKEN: 
  CLAUDE_CODE_OAUTH_TOKEN: 
  COPILOT_GITHUB_TOKEN: 
  GOOGLE_API_KEY: 
  GEMINI_API_KEY: 

All secrets are empty strings printed at job initialization.

Pattern 2 — Mass cancellations (05:05–05:10 window)

No error log available — cancelled runs never reach a step.

Root cause (inferred): Dependabot pushed a batch of version-bump PRs (#309, #311, others) nearly simultaneously, spawning ~50 workflow runs in ~5 minutes. With cancel-in-progress: false, runs queue but do not cancel each other within the same concurrency group. However, as Dependabot PRs auto-merge or are superseded, GitHub Actions cancels any pending/queued runs associated with the now-closed PRs before a runner is assigned. This is consistent with the interleaved pattern (e.g., run #1147 succeeded at 05:05:29Z while #1145 and #1145 were cancelled at the same second — different PRs sharing the same concurrency slot).

---

4. Token Scope Analysis

Scopes currently present (from workflow + successful runs):

Token	Source	Used For
GH_TOKEN → DON_PETRY_BOT_GH_PAT	Repo secret	All gh CLI calls, PR review submission, list-prs.sh
CLAUDE_CODE_OAUTH_TOKEN	Repo secret	Claude Code CLI authentication
COPILOT_GITHUB_TOKEN → GH_PAT	Repo secret	gh-copilot fallback
GOOGLE_API_KEY	Repo secret	Gemini CLI (fallback engine)
GITHUB_TOKEN (implicit)	Actions runtime	actions/checkout only

Scopes missing or inaccessible in Dependabot context:

Secret	Status in Dependabot Runs	Impact
DON_PETRY_BOT_GH_PAT	Empty — not in Dependabot secrets store	Job fails at gh auth status; no review attempted
CLAUDE_CODE_OAUTH_TOKEN	Empty — not in Dependabot secrets store	Claude CLI would fail even if gh auth passed
COPILOT_GITHUB_TOKEN	Empty — not in Dependabot secrets store	Fallback engine unavailable
GOOGLE_API_KEY	Empty — not in Dependabot secrets store	Second fallback engine unavailable

Recommendations per missing scope:

• DON_PETRY_BOT_GH_PAT — Add to Settings → Secrets and variables → Dependabot. This is the minimum required to unblock the job. The bot reviewing Dependabot PRs is a valid use case (dependency upgrades need review too).
• CLAUDE_CODE_OAUTH_TOKEN — Add to Dependabot secrets. Required for the primary review engine to authenticate.
• COPILOT_GITHUB_TOKEN and GOOGLE_API_KEY — Add to Dependabot secrets if fallback engines should be available for Dependabot PRs; otherwise the primary engine fix is sufficient.

---

5. Recommendations

1. Add critical secrets to Dependabot secrets store [CRITICAL]

• What: In GitHub repo settings → Settings → Secrets and variables → Dependabot, add secrets: DON_PETRY_BOT_GH_PAT, CLAUDE_CODE_OAUTH_TOKEN, COPILOT_GITHUB_TOKEN, GOOGLE_API_KEY.
• Why: Dependabot-triggered workflow runs cannot access standard repository secrets; they silently become empty strings, causing gh auth status to fail and killing the job before any review logic runs.
• Expected impact: Eliminates all current failures (100% of failures caused by this). Dependabot PRs will be reviewed normally.
• Urgency: CRITICAL — current failure rate for Dependabot PRs is 100%.

2. Add a Dependabot guard as defense-in-depth [HIGH]

• What: In .github/workflows/pr-review.yml, extend the job-level if condition or add an explicit early-exit step:

  # Option A: skip entirely (if reviewing Dependabot PRs is not desired)
  if: >-
    github.actor != 'dependabot[bot]'
    && (... existing conditions ...)
  
  # Option B: emit a clear warning instead of a cryptic auth error
  - name: Check secret availability
    run: |
      if [ -z "${GH_TOKEN}" ]; then
        echo "::warning::GH_TOKEN is empty — likely a Dependabot context. Add DON_PETRY_BOT_GH_PAT to Dependabot secrets."
        exit 0  # or exit 1 to keep it as failure for visibility
      fi

• Why: Even after adding secrets to the Dependabot store, a guard prevents confusing silent failures if a secret is later rotated or removed, and gives an actionable message rather than a generic gh auth error.
• Expected impact: Clearer failure messages; prevents future recurrence from misconfigured secrets.
• Urgency: HIGH — should accompany recommendation test issue from agent #1.

3. Investigate and document the 41% cancellation rate [MEDIUM]

• What: Add logging to determine cancellation source. Consider adding a workflow_run status notification step, or use the GitHub API to query cancellation reasons. Also review whether cancel-in-progress: false is the correct setting: if Dependabot batch-updates 20 PRs and each gets two triggers (synchronize + check_suite), 40 runs queue per batch. Setting cancel-in-progress: true on the pr-review-batch group (while keeping it false on per-PR groups) would reduce queue buildup.
• Why: A 41% cancellation rate wastes queued runner slots and makes the run history noisy. If cancellations are truly benign (PR merged before run starts), the rate is acceptable but should be confirmed.
• Expected impact: Cleaner run history; reduced unnecessary runner provisioning.
• Urgency: MEDIUM — not causing failures, but degrades observability.

4. Pin Dependabot secret rotation to workflow secret rotation [LOW]

• What: Add a note in the repo's runbook or a workflow comment that DON_PETRY_BOT_GH_PAT must be updated in both the repository secrets store and the Dependabot secrets store whenever the PAT is rotated.
• Why: Dependabot secrets are a separate namespace; they are easy to forget during PAT rotation, which would silently re-introduce this exact failure.
• Expected impact: Prevents recurrence after the next PAT rotation.
• Urgency: LOW — operational hygiene.

---

6. Health Score

Health: 7/10 — Core review engine is working correctly for non-Dependabot PRs, but a single missing secrets configuration causes 100% failure on all Dependabot-triggered runs; fix is mechanical and well-understood.