-
Notifications
You must be signed in to change notification settings - Fork 1
fix(health-check): pin claude-code version via cache, surface CLI errors #175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
da2697d
fix(health-check): pin claude-code version via cache, surface CLI errors
don-petry f21e7f6
fix(health-check): suppress SonarCloud false positives on npm install
don-petry c03aceb
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 96c1657
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 97cc9ce
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 11a9957
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry cfec9e7
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry a34c85e
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry d218621
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry e3ded43
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 618ec12
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 3a59b7a
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry b44ad43
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry b771883
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry e98239b
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 02acc22
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 3750633
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 7679515
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 7d2516a
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 5ae6046
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 0a63398
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry f148b0a
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 72cf436
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 753c7eb
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry dfe737b
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry a2b36d0
Merge branch 'main' into fix/health-check-claude-version-skew
don-petry 91ead79
fix(sonar): exclude .github/workflows from SonarCloud analysis
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| name: Daily PR Review Health Check | ||
|
|
||
| on: | ||
| schedule: | ||
| # Daily at 6:00 UTC — covers same-day run history before US work hours | ||
| - cron: '0 6 * * *' | ||
| workflow_dispatch: | ||
| inputs: | ||
| lookback_days: | ||
| description: "Days of run history to inspect (time-based search)" | ||
| required: false | ||
| default: "1" | ||
| type: string | ||
|
|
||
| permissions: | ||
| contents: read | ||
| issues: write | ||
|
|
||
| concurrency: | ||
| group: daily-pr-review-health | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| health-check: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| env: | ||
| CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} | ||
| LOOKBACK_DAYS: ${{ inputs.lookback_days || '1' }} | ||
| # Pin via vars.CLAUDE_CODE_VERSION for reproducible caching; default | ||
| # 'latest' caches an install indefinitely — flush via the Actions cache | ||
| # UI or bump the variable to pick up new releases. Mirrors pr-review.yml. | ||
| CLAUDE_CODE_VERSION: ${{ vars.CLAUDE_CODE_VERSION || 'latest' }} | ||
|
|
||
| steps: | ||
| - name: Checkout agent repo | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
|
|
||
| - name: Set up Node | ||
| uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 | ||
| with: | ||
| node-version: '20' | ||
|
|
||
| - name: Cache claude-code CLI | ||
| uses: actions/cache@v5 | ||
| with: | ||
| path: ~/.npm-global | ||
| key: claude-code-${{ env.CLAUDE_CODE_VERSION }}-${{ runner.os }} | ||
|
|
||
| - name: Install Claude Code CLI | ||
| run: | | ||
| mkdir -p "$HOME/.npm-global" | ||
| npm config set prefix "$HOME/.npm-global" | ||
| echo "$HOME/.npm-global/bin" >> "$GITHUB_PATH" | ||
| export PATH="$HOME/.npm-global/bin:$PATH" | ||
| if command -v claude >/dev/null 2>&1; then | ||
| echo "claude-code cache hit: $(claude --version)" | ||
| else | ||
| npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" # NOSONAR — install scripts required for native binary setup; version intentionally uses CLAUDE_CODE_VERSION var (same pattern as pr-review.yml) | ||
| fi | ||
|
|
||
| - name: Run health check | ||
| env: | ||
| GH_TOKEN: ${{ secrets.DON_PETRY_BOT_GH_PAT }} | ||
| run: bash scripts/pr_review_health.sh | ||
|
|
||
| - name: Truncate report if needed | ||
| run: | | ||
| if [ -f pr_review_health_report.md ]; then | ||
| if [ "$(wc -c < pr_review_health_report.md)" -gt 60000 ]; then | ||
| head -c 60000 pr_review_health_report.md > pr_review_health_report.md.tmp | ||
| mv pr_review_health_report.md.tmp pr_review_health_report.md | ||
| printf '\n\n---\n_Report truncated at 60 000 bytes._\n' >> pr_review_health_report.md | ||
| fi | ||
| fi | ||
|
|
||
| - name: Create issue with findings | ||
| if: env.HAS_FAILURES == 'true' && hashFiles('pr_review_health_report.md') != '' | ||
| uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 | ||
| with: | ||
| github-token: ${{ github.token }} | ||
| script: | | ||
| const fs = require('fs'); | ||
| const report = fs.readFileSync('pr_review_health_report.md', 'utf8'); | ||
| if (!report.trim()) { | ||
| core.setFailed('Report file is empty'); | ||
| return; | ||
| } | ||
| const today = new Date().toISOString().split('T')[0]; | ||
| const issue = await github.rest.issues.create({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| title: `PR Review Agent — failures detected ${today}`, | ||
| body: report, | ||
| labels: ['health-check', 'automated-report'], | ||
| }); | ||
| core.notice(`Issue created: ${issue.data.html_url}`); | ||
|
|
||
| - name: Annotate clean run | ||
| if: env.HAS_FAILURES != 'true' | ||
| run: | | ||
| echo "::notice::Health check passed — all runs in the last ${LOOKBACK_DAYS} day(s) are healthy." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,234 @@ | ||
| #!/usr/bin/env bash | ||
| # Daily health check for the PR Review Agent workflow. | ||
| # | ||
| # Fetches recent pr-review.yml run logs, feeds them to Claude for pattern | ||
| # analysis, and writes a markdown report to pr_review_health_report.md. | ||
| # Sets HAS_FAILURES=true in $GITHUB_ENV when failed runs are detected. | ||
| # | ||
| # Env vars consumed: | ||
| # GH_TOKEN — primary (GitHub App; must have actions:read on this repo) | ||
| # GH_PAT_FALLBACK — fallback PAT if App token lacks access | ||
| # CLAUDE_CODE_OAUTH_TOKEN — passed through to claude CLI | ||
| # LOOKBACK_DAYS — days of history to consider (default: 1, ~24 hours) | ||
| # Set higher to examine longer windows (e.g., LOOKBACK_DAYS=7 for weekly review) | ||
| # GITHUB_ENV — written by Actions runner; used to export HAS_FAILURES | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| LOOKBACK_DAYS="${LOOKBACK_DAYS:-1}" | ||
| WORKFLOW_REPO="${AGENT_REPO:-petry-projects/.github-private}" | ||
| WORKFLOW_FILE="pr-review.yml" | ||
| REPORT_FILE="pr_review_health_report.md" | ||
| LOG_DIR="health_run_logs" | ||
| TODAY=$(date -u +%Y-%m-%d) | ||
|
|
||
| echo "=== PR Review Agent — Daily Health Check ===" | ||
| echo " Repo: $WORKFLOW_REPO" | ||
| echo " Workflow: $WORKFLOW_FILE" | ||
| echo " Lookback: ${LOOKBACK_DAYS} day(s)" | ||
| echo " Date: $TODAY" | ||
| echo "" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 0. Token selection — verify GH_TOKEN has access to workflow run logs | ||
| # --------------------------------------------------------------------------- | ||
| if ! gh api "repos/${WORKFLOW_REPO}/actions/workflows/${WORKFLOW_FILE}/runs?per_page=1" \ | ||
| >/dev/null 2>&1; then | ||
| if [ -n "${GH_PAT_FALLBACK:-}" ]; then | ||
| echo "::warning::App token cannot access ${WORKFLOW_REPO} run logs — using GH_PAT_FALLBACK" | ||
| export GH_TOKEN="$GH_PAT_FALLBACK" | ||
| else | ||
| echo "::error::App token cannot access ${WORKFLOW_REPO} run logs and GH_PAT_FALLBACK is not set." | ||
| echo "::error::Grant the GitHub App access to ${WORKFLOW_REPO} or set the DON_PETRY_BOT_GH_PAT secret." | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 1. Fetch recent run metadata | ||
| # --------------------------------------------------------------------------- | ||
| # GNU date: date -d "N days ago"; macOS: date -v-Nd | ||
| CUTOFF=$(date -u -d "${LOOKBACK_DAYS} days ago" +%Y-%m-%dT%H:%M:%SZ 2>/dev/null \ | ||
| || date -u -v-"${LOOKBACK_DAYS}"d +%Y-%m-%dT%H:%M:%SZ) | ||
|
|
||
| echo "Fetching all runs since: $CUTOFF" | ||
| # Fetch with per_page=100 (GitHub API max) to capture all runs in the lookback window. | ||
| # Time-based filtering ensures we get every run created at or after CUTOFF, regardless of quantity. | ||
| runs_json=$(gh api \ | ||
| "repos/${WORKFLOW_REPO}/actions/workflows/${WORKFLOW_FILE}/runs?per_page=100&created=>=${CUTOFF}" \ | ||
| --jq '.workflow_runs | map({ | ||
| id: .id, | ||
| status: .status, | ||
| conclusion: .conclusion, | ||
| created_at: .created_at, | ||
| html_url: .html_url, | ||
| run_number: .run_number | ||
| })' 2>/dev/null || echo '[]') | ||
|
|
||
| read -r total_runs failed_runs success_runs cancelled_runs < <(echo "$runs_json" | jq -r ' | ||
| [ | ||
| length, | ||
| ([.[] | select(.conclusion == "failure")] | length), | ||
| ([.[] | select(.conclusion == "success")] | length), | ||
| ([.[] | select(.conclusion == "cancelled")] | length) | ||
| ] | @tsv') | ||
|
|
||
| echo " Total runs: $total_runs" | ||
| echo " Successful: $success_runs" | ||
| echo " Failed: $failed_runs" | ||
| echo " Cancelled: $cancelled_runs" | ||
| echo "" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 2. Early exit when no failures | ||
| # --------------------------------------------------------------------------- | ||
| if [ "$failed_runs" -eq 0 ]; then | ||
| echo "No failed runs in the last ${LOOKBACK_DAYS} days. Health check passed." | ||
| printf '# PR Review Agent Health Check — %s\n\nAll %d run(s) inspected over the last %d days succeeded. No action required.\n' \ | ||
| "$TODAY" "$total_runs" "$LOOKBACK_DAYS" > "$REPORT_FILE" | ||
| exit 0 | ||
| fi | ||
|
|
||
| # Export flag for workflow step condition | ||
| [ -n "${GITHUB_ENV:-}" ] && echo "HAS_FAILURES=true" >> "$GITHUB_ENV" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 3. Download logs for failed runs | ||
| # --------------------------------------------------------------------------- | ||
| mkdir -p "$LOG_DIR" | ||
| failed_run_ids=$(echo "$runs_json" | jq -r '.[] | select(.conclusion == "failure") | .id') | ||
|
|
||
| for run_id in $failed_run_ids; do | ||
| { | ||
| gh run view "$run_id" --repo "$WORKFLOW_REPO" --log 2>/dev/null \ | ||
| | head -c 200000 \ | ||
| > "${LOG_DIR}/run_${run_id}.txt" \ | ||
| || echo "(log unavailable for run $run_id)" > "${LOG_DIR}/run_${run_id}.txt" | ||
| } & | ||
| done | ||
| wait | ||
|
|
||
| # Surface log-retrieval failures so the operator knows the diagnosis may be incomplete. | ||
| missing_logs=0 | ||
| for run_id in $failed_run_ids; do | ||
| if grep -q "^(log unavailable" "${LOG_DIR}/run_${run_id}.txt" 2>/dev/null; then | ||
| echo "::warning::Failed run $run_id: log could not be retrieved — diagnosis will be incomplete for this run" | ||
| missing_logs=$((missing_logs + 1)) | ||
| fi | ||
| done | ||
| if [ "$missing_logs" -gt 0 ]; then | ||
| echo " $missing_logs of $failed_runs failed run log(s) could not be retrieved" | ||
| fi | ||
| echo "" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 4. Fetch workflow source for token/permission context | ||
| # --------------------------------------------------------------------------- | ||
| echo "Fetching ${WORKFLOW_FILE} source..." | ||
| workflow_source=$(gh api \ | ||
| "repos/${WORKFLOW_REPO}/contents/.github/workflows/${WORKFLOW_FILE}" \ | ||
| --jq '.content' 2>/dev/null | base64 -d 2>/dev/null || echo "(workflow source unavailable)") | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 5. Build analysis prompt and invoke Claude | ||
| # --------------------------------------------------------------------------- | ||
| RUNS_SUMMARY=$(echo "$runs_json" | jq -r \ | ||
| '.[] | "[\(.conclusion // "unknown")] run #\(.run_number) (\(.created_at)) — \(.html_url)"') | ||
|
|
||
| logs_file=$(mktemp) | ||
| # One jq pass over failed runs; append each log file in the same order | ||
| while IFS=$'\t' read -r run_id run_meta; do | ||
| log_file="${LOG_DIR}/run_${run_id}.txt" | ||
| [ -f "$log_file" ] || continue | ||
| { | ||
| printf '=== LOG: %s ===\n' "$run_meta" | ||
| cat "$log_file" | ||
| printf '=== END LOG ===\n\n' | ||
| } >> "$logs_file" | ||
| done < <(echo "$runs_json" | jq -r \ | ||
| '.[] | select(.conclusion == "failure") | [(.id | tostring), "run #\(.run_number) (\(.conclusion)) at \(.created_at)"] | @tsv') | ||
|
|
||
| echo "Invoking Claude for log analysis..." | ||
| # Claude writes errors to stdout (not stderr), so they'd silently land in | ||
| # $REPORT_FILE with the stdout redirect below. Wrap in `if !` so a non-zero | ||
| # exit surfaces the file contents to the Actions log before aborting. | ||
| if ! claude --print --model claude-sonnet-4-6 --no-session-persistence > "$REPORT_FILE" <<PROMPT | ||
| You are analyzing GitHub Actions workflow run logs for the PR Review Agent. | ||
|
|
||
| ## Context | ||
| - Workflow: \`${WORKFLOW_FILE}\` in repo \`${WORKFLOW_REPO}\` | ||
| - Analysis window: last ${LOOKBACK_DAYS} days, up to 100 most recent runs | ||
| - Report date: ${TODAY} | ||
| - Total runs fetched: ${total_runs} | Successful: ${success_runs} | Failed: ${failed_runs} | Cancelled: ${cancelled_runs} | ||
|
|
||
| ## Run Summary | ||
| ${RUNS_SUMMARY} | ||
|
|
||
| ## Workflow Source (.github/workflows/${WORKFLOW_FILE}) | ||
| \`\`\`yaml | ||
| ${workflow_source} | ||
| \`\`\` | ||
|
|
||
| ## Failed Run Logs | ||
| $(cat "$logs_file") | ||
|
|
||
| --- | ||
|
|
||
| Analyze these logs and produce a markdown health report with the following sections: | ||
|
|
||
| ### 1. Executive Summary | ||
| Use F-style layout — lead with the most critical signal, then supporting bullets. No prose paragraphs. | ||
|
|
||
| Format: | ||
| **Status:** BLOCKING | DEGRADED | WARNING | HEALTHY | ||
| **Period:** <date range> | ||
| **Result:** <X of Y runs failed (Z%)> | ||
|
|
||
| Key findings: | ||
| - <dominant failure cause — one line> | ||
| - <secondary issue if any — one line> | ||
| - <any pattern worth noting — one line> | ||
|
|
||
| Action required: <one imperative sentence, or "None" if healthy> | ||
|
|
||
| ### 2. Failure Breakdown | ||
| A table with columns: Failure Category | Affected Runs | Example Error Message. | ||
| Categories to look for (not exhaustive): | ||
| - CLI breaking change (e.g. invalid flag values) | ||
| - Permission / auth error (403, 401, insufficient scope) | ||
| - GitHub API rate limit | ||
| - Missing token scope (e.g. read:org, read:packages) | ||
| - Engine rate limit (Claude or Copilot quota) | ||
| - Timeout / infrastructure | ||
| - Other / unknown | ||
|
|
||
| ### 3. Error Patterns | ||
| For each category found: quote the exact error message from the logs, identify which step and script it comes from, and explain the root cause. | ||
|
|
||
| ### 4. Token Scope Analysis | ||
| From the workflow source and any gh auth status output in the logs, list: | ||
| - Scopes currently present | ||
| - Scopes that appear missing or insufficient based on the errors | ||
| - Recommendation for each missing scope | ||
|
|
||
| ### 5. Recommendations | ||
| Numbered list. For each issue include: what to change (file, line, command), why, expected impact after fix, and urgency: [CRITICAL | HIGH | MEDIUM | LOW]. | ||
| Mark CRITICAL if the issue causes 100% workflow failure. | ||
|
|
||
| ### 6. Health Score | ||
| Single line: \`Health: X/10 — <one-sentence verdict>\` | ||
| (10 = all runs passing; 0 = complete outage) | ||
|
|
||
| Output ONLY the markdown report — no preamble or commentary outside the report sections. | ||
| PROMPT | ||
| then | ||
| echo "::error::Claude invocation failed. CLI output follows:" | ||
| cat "$REPORT_FILE" >&2 | ||
| rm -f "$logs_file" | ||
| exit 1 | ||
| fi | ||
| rm -f "$logs_file" | ||
|
|
||
| echo "" | ||
| echo "Report written to $REPORT_FILE ($(wc -c < "$REPORT_FILE") bytes)" | ||
| echo "=== Health check complete ===" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pin actions/cache to a SHA for consistency and security.
All other actions in this workflow are pinned to commit SHAs (lines 37, 40, 79), but
actions/cache@v5is not. This creates a security and reproducibility gap.📌 Proposed fix to pin actions/cache
🤖 Prompt for AI Agents