fix(fleet-monitor): auto-close resolved fleet-tracker issues + stop daily-digest accumulation (#1203) - #1204
Conversation
The monitor creates/updates a fleet-tracker issue per (repo × workflow) above 10% failure, but never closed them when the workflow recovered or was deleted — so 107 stale alerts accumulated (some with 'Last updated' stamps ~55 days old while the monitor runs daily; 4 for the deleted claude.yml #456). Add a step that closes any open fleet-tracker issue whose 'Last updated on <date>' stamp is older than STALE_DAYS (default 3 = 3 missed daily runs), with a ✅ Auto-resolved comment. It reopens automatically (fresh issue) if the workflow crosses the threshold again — same lifecycle as org-scorecard. Safety: gated on fleet_high_failure.json existing (a failed scan can't mass-close); staleness uses each issue's own stamp, not the current run, so a single bad-scan day never false-closes; comment + close are each guarded so a comment-capped issue still closes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Juznz5V6su81ffSND8fg7s
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
|
Warning Review limit reached
Next review available in: 7 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe Fleet Monitor workflow adds a conditional step that identifies stale managed issues using repository matches and body timestamps, comments on qualifying issues, closes them as completed, and reports closed, kept, and skipped counts. ChangesFleet tracker resolution
Estimated code review effort: 3 (Moderate) | ~15 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #1204 |
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/actions-fleet-monitor.yml:
- Around line 371-372: Update the token configuration in the workflow step using
the github-token field to reference the GH_PAT_WORKFLOWS environment variable
rather than accessing secrets.GH_PAT_WORKFLOWS directly. Preserve the existing
cross-repository issue-writing behavior.
- Line 366: Update the actions/github-script step’s GH_PAT_WORKFLOWS reference
to use the workflow environment variable env.GH_PAT_WORKFLOWS instead of
secrets.GH_PAT_WORKFLOWS, preserving the existing action configuration.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 38666435-62f4-4c72-aef0-8d1b258a0810
📒 Files selected for processing (1)
.github/workflows/actions-fleet-monitor.yml
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #1204 |
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
|
Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-07-14T04:04:36Z. |
don-petry
changed the title
… them The 'workflow failures detected <date>' digest embeds the date in its title, so issues.create ran unconditionally every day — 42 open digests had piled up. Close the prior open digests (health-check label, matching title prefix) before opening today's, so only the current snapshot stays open. Complements the fleet-tracker auto-close in this PR; both stop the monitor accumulating stale issues. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Juznz5V6su81ffSND8fg7s
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
Dev-Lead — fix-bot-comment (applied)Changes committed and pushed. |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: 12e06099cb911868f3bd7b22a8d4029727d41dd0
Review mode: triage-approved (single reviewer)
Summary
Adds an auto-close path to the Actions Fleet Monitor: (1) a new 'Close resolved fleet-tracker issues' step that closes any open fleet-tracker issue whose per-issue 'Last updated' stamp is older than STALE_DAYS=3, with an auto-resolved comment; (2) closes prior open daily-digest issues before creating today's. Fixes the 107-stale-alert backlog described in #1203. Implementation reuses the file's established patterns (same pinned github-script v9.0.0 SHA, same hashFiles gate as the tracking step, GH_PAT_WORKFLOWS via env per repo guidelines) and the stamp regex matches the exact format the tracking step writes.
Linked issue analysis
Linked issue #1203 (fleet-tracker issues never auto-close on recovery/deletion) is substantively addressed: recovered workflows stop being stamped and are closed after 3 missed daily runs; deleted workflows (never re-scanned) age out the same way. Safety properties claimed in the issue are present in the code: step gated on fleet_high_failure.json existing (failed scan cannot mass-close), staleness derived from each issue's own stamp (a single bad-scan day cannot false-close), comment and close individually try/catch'd.
Findings
No blocking findings.
- Prior reviewer threads all resolved at this head: CodeRabbit's token-sourcing guideline (github-token now from env.GH_PAT_WORKFLOWS) and Copilot's github.paginate Search-API mapping bug (response => response.data.items now present). CodeRabbit's final review is APPROVED.
- Security: action pinned to the same SHA already used 4x in this file; no new secrets or permissions — GH_PAT_WORKFLOWS cross-repo issues:write is this workflow's existing, documented pattern. Issue bodies are only regex-matched (strict date capture), never executed or interpolated unsafely. Scheduled trigger, no PR-controlled inputs.
- Secret scan: run_secret_scanning MCP tool not available in this environment; gitleaks CI check is green and the diff introduces no secret material.
- Minor (non-blocking): search.issuesAndPullRequests is a legacy search endpoint; fine today, may warrant migration if GitHub deprecates it.
CI status
All substantive checks green: CodeQL, gitleaks, ShellCheck, Lint, SonarCloud, agent-shield, Agent Security Scan, unit-tests, bats, holdout-guard, template-drift, gh-aw-compile, validate-agent-profiles/personas. The two cancelled dev-lead jobs are superseded concurrency runs of the agent-orchestration workflow (latest run pending, not a correctness gate); review/review in-progress is this review run itself. mergeStateStatus BLOCKED is due to the required org-leads review.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #1204 |
|
Note @don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically. |
|
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: a296f3a6f0eeec84820da193ed429a979d1637f6
Review mode: triage-approved (single reviewer)
Summary
Adds lifecycle management to the Actions Fleet Monitor: (1) a new 'Close resolved fleet-tracker issues' step that closes org-wide fleet-tracker issues whose per-issue 'Last updated' stamp is older than STALE_DAYS (3), with an auto-resolved comment; (2) closes prior daily-digest issues before creating today's so only one snapshot stays open; (3) re-plumbs the existing GH_PAT_WORKFLOWS secret through env: blocks per org coding guidelines. Since the prior automated approval at 12e0609, the only changes are the token-plumbing fix (addressing CodeRabbit's guideline comment) and a merge of main — no behavioral change to the reviewed logic.
Linked issue analysis
PR references issue #1203 (fleet-tracker issues never auto-close on recovery/deletion, 107 stale open alerts). The new auto-close step directly implements the fix described in the issue, with the stated safety gates: gated on fleet_high_failure.json existing (failed scan cannot mass-close), staleness computed from each issue's own stamp (single bad-scan day cannot false-close), and per-issue try/catch on comment+close. The digest-accumulation half (42 open dated digest issues) is also addressed. Issue is substantively resolved.
Findings
No blocking findings. Verified: (1) github-script pinned to full SHA (v9.0.0), unchanged; (2) no script injection — scripts read ORG/STALE_DAYS via process.env, no untrusted ${{ }} interpolation inside script bodies; (3) search.issuesAndPullRequests pagination uses the response.data.items mapper (Copilot's earlier finding, resolved); (4) stamp regex only matches monitor-managed bodies, unmatched issues are skipped; (5) all 3 review threads (2 CodeRabbit, 1 Copilot) are resolved and both bot change-requests were dismissed after fixes. Secret-scanning MCP tool unavailable in this session — relied on the green gitleaks CI check; no secrets appear in the diff (existing secret re-plumbed only).
CI status
All required checks green (SonarCloud, CodeQL, agent-shield/AgentShield, dependency-audit/Detect ecosystems), plus Lint, ShellCheck, shellcheck, unit-tests, bats, gitleaks, Agent Security Scan, gh-aw-compile, template-drift, holdout-guard, validate-agent-profiles, validate-personas, CodeRabbit all SUCCESS. dev-lead/dispatch and dev-lead/ci-relay are CANCELLED — non-required relay jobs intentionally skipped via [skip ci-relay] in the head commit. reviewDecision: APPROVED.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: a296f3a6f0eeec84820da193ed429a979d1637f6
Review mode: triage-approved (single reviewer)
Summary
Adds a stale-based auto-close step for cross-repo fleet-tracker issues and closes prior daily-digest issues before creating today's, fixing the 107-stale-alert backlog described in #1203. Also re-plumbs the existing GH_PAT_WORKFLOWS secret through step-level env per org guidelines (functionally equivalent, no new exposure).
Linked issue analysis
Issue #1203 (fleet-tracker issues never auto-close on recovery/deletion → 107 stale alerts) is substantively addressed. The new "Close resolved fleet-tracker issues" step implements exactly the fix the issue describes: close any open fleet-tracker issue whose per-issue "Last updated ... on " stamp is older than STALE_DAYS (3), with an auto-resolved comment. Deleted workflows are covered by the same staleness path; the 4 deleted-claude.yml issues are being hand-closed per the PR description. The digest de-dup change additionally stops the date-titled daily digest from accumulating (closes prior digests before opening today's).
Findings
- Safety gating is sound: the close step is gated on
hashFiles('fleet_high_failure.json') != ''so a failed scan cannot mass-close, and staleness uses each issue's own stamp, so a single bad-scan day cannot false-close. Comment and close are independently try/catch'd. - Correct Search API pagination:
github.paginate(search.issuesAndPullRequests, ..., response => response.data.items)uses the mapping function — the earlier Copilot finding on this was addressed (thread resolved). - Token handling:
GH_PAT_WORKFLOWSmoved from directsecrets.reference inwith:to step-levelenvper CodeRabbit's org-guideline comment (thread resolved). Same token, same scope, same steps — no new auth surface; the new step reuses the identical cross-repo issues:write pattern already established in this workflow. Action remains pinned to the full SHA (v9.0.0). - No injection surface: scripts read only
process.envand API data; untrusted issue bodies/titles are only regex-matched, never interpolated into expressions or commands. - Minor (non-blocking): if the whole fleet recovers, the digest-creation step won't run, so the last daily digest stays open until failures recur — at most one open digest, consistent with the "current snapshot stays open" design.
- Secret-scanning MCP tool unavailable in this environment; gitleaks CI check is green and the diff contains only
${{ secrets.* }}references, no literal credentials.
CI status
All required checks green: SonarCloud ✓, CodeQL ✓, agent-shield / AgentShield ✓, dependency-audit / Detect ecosystems ✓. Also green: Lint, shellcheck/ShellCheck, unit-tests, bats, gitleaks, CodeRabbit, Agent Security Scan, gh-aw-compile, template-drift, holdout-guard. The cancelled dev-lead / dispatch and dev-lead / ci-relay runs are non-required agent-orchestration relays superseded by newer runs; review / review (in progress) is this review. All review threads resolved; reviewDecision is APPROVED.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
…close (#1242) A `response => response.data.items` mapFn added to the fleet-tracker auto-close paginate call (during #1204 review) produced undefined entries, so the step threw 'TypeError: Cannot read properties of undefined (reading repository_url)' and failed the entire daily monitor — the fleet-tracker/health backlog stopped clearing. octokit already flattens search results to .items natively; remove the redundant mapFn and add a defensive .filter(Boolean). Claude-Session: https://claude.ai/code/session_01Juznz5V6su81ffSND8fg7s Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Don Petry Bot <donpetry+bot@gmail.com>
The 'Summarize token usage by workflow' step fetches/unzips every token-usage artifact and hit an API rate-limit (HTTP 403), failing the whole daily monitor BEFORE the load-bearing steps ran — issue tracking + the fleet-tracker auto-close (#1204) never executed, so the backlog didn't clear even after the paginate fix (#1242). This step is best-effort observability; mark it continue-on-error so a transient rate-limit can never again abort the monitor and block the auto-close. Claude-Session: https://claude.ai/code/session_01Juznz5V6su81ffSND8fg7s Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Problem
The Actions Fleet Monitor creates/updates a
fleet-trackerissue per (repo × workflow) above 10% failure, but never closes it when the workflow recovers (≤10%) or is deleted. Result: 107 stale open alerts, some stampedLast updated~55 days ago while the monitor runs daily, plus 4 for theclaude.ymlworkflow deleted in #456. Full write-up: #1203.Fix
New "Close resolved fleet-tracker issues" step: closes any open
fleet-trackerissue whoseLast updated ... on <date>stamp is older thanSTALE_DAYS(default 3 = 3 missed daily runs), with a✅ Auto-resolvedcomment. Reopens automatically (fresh issue) if the workflow crosses the threshold again — same lifecycle asorg-scorecard.yml.Safety
fleet_high_failure.jsonexisting → a failed scan can't mass-close.try/catch— a comment-capped issue still closes.github-script@v9.0.0+GH_PAT_WORKFLOWS(cross-repoissues:write).Validation
node --checkon the extracted script passes; yamllint clean under the repo's exact config (max 250).Rollout
On merge, the next daily run auto-closes the recovered/deleted backlog. The 4 deleted-
claude.ymlissues are being closed immediately by hand (a deleted workflow is never re-scanned).🤖 Generated with Claude Code
https://claude.ai/code/session_01Juznz5V6su81ffSND8fg7s
Summary by CodeRabbit