feat(aw): make CI Failure Analyst org-wide via reusable workflow - #367
Merged
Conversation
|
Warning Review limit reached
Your plan currently allows 1 review/hour. Refill in 22 minutes and 47 seconds. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more review capacity refills, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than trial, open-source, and free plans. In all cases, review capacity refills continuously over time. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Collaborator
Author
|
No description provided. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request transitions the CI Failure Analyst to a reusable workflow model triggered by check_run events, updating documentation and providing a template stub for deployment. Feedback identifies a consistency issue between the documentation and test scenarios regarding idempotency behavior (skipping vs. updating) and suggests pinning the reusable workflow to a specific commit SHA instead of the @main branch to ensure stability.
Collaborator
Author
|
No description provided. |
Contributor
There was a problem hiding this comment.
Pull request overview
Extracts the CI Failure Analyst workflow into a workflow_call reusable so it can be adopted org-wide via a thin caller stub, mirroring the established reusable-workflow deployment approach used elsewhere in this repo.
Changes:
- Added a reusable workflow (
ci-failure-analyst-reusable.yml) containing the full analysis + commenting logic. - Replaced the repo-local workflow with a thin stub (
ci-failure-analyst.lock.yml) that calls the reusable. - Added an adoption template (
templates/ci-failure-analyst.yml) and updated documentation to reflect the new architecture.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
templates/ci-failure-analyst.yml |
New copy/paste stub for other repos to call the reusable workflow. |
docs/aw/ci-failure-analyst.md |
Updated docs for reusable architecture, trigger behavior, and deployment steps. |
.github/workflows/ci-failure-analyst.lock.yml |
Converted to a thin caller stub for this repo. |
.github/workflows/ci-failure-analyst-reusable.yml |
New reusable workflow containing PR resolution, idempotency, analysis, and comment posting. |
The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks.
$ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive.
Collaborator
Author
|
No description provided. |
donpetry-bot
previously approved these changes
May 23, 2026
donpetry-bot
left a comment
donpetry-bot
left a comment
Contributor
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 24b0e5f6fe9391195ef88063c74b7ac1571b9c5b
Review mode: triage-approved (single reviewer)
Summary
This PR refactors the CI Failure Analyst from a monolithic in-repo workflow into a workflow_call reusable + thin caller stub, plus an adoption template for other org repos. It mirrors the established dev-lead-reusable.yml pattern. The analysis logic is moved largely 1:1 (Claude Code invocation, idempotency check, prompt template) with small, reasonable improvements: an explicit anti-loop guard (!startsWith(check_run.name, 'CI Failure Analyst')), a non-fork PR filter in the resolution step, and refreshed docs.
Linked issue analysis
No linked issue; the PR body itself motivates the change (enable org-wide adoption without duplicating ~150 lines per repo). The architecture choice (reusable vs. repository_dispatch) is well-justified in the PR description — read-only diagnostic comment using the caller's own GITHUB_TOKEN, so no GH_PAT_WORKFLOWS relay job is needed.
Findings
No blocking issues.
Nit / follow-up (non-blocking): templates/ci-failure-analyst.yml:43 pins uses: to commit 69e9774818d45846f3a850ca13f31c7e9d6345cc, which is the prior main HEAD (merge of #357). The reusable workflow file does not exist at that SHA — it is introduced by this PR. Anyone who copies the template before the SHA is bumped post-merge would get a workflow that fails to resolve the reusable. Either bump the SHA in a follow-up commit immediately after merge, or accept the slight risk that this template is unused until then. (The in-repo lock.yml correctly uses @main and is unaffected.)
Observations (informational):
- Anti-loop guard
!startsWith(github.event.check_run.name, 'CI Failure Analyst')is a sensible new safeguard. Naming-wise, the reusable workflow is"CI Failure Analyst — Reusable"and the lock workflow is"CI Failure Analyst", so both are caught by the prefix match. - Idempotency now intentionally skips (rather than updating) when a
<!-- ci-analyst sha=... -->comment already exists. The scenario spec attests/aw/ci-failure-analyst/scenarios.mdwas updated in the same PR to match — good consistency. - The
shellcheck disable=SC2016comment at.github/workflows/ci-failure-analyst-reusable.yml:33is legitimate —$ENV.REPOinside a single-quoted jq filter is jq's environment-variable syntax, not a shell expansion. - Prompt-injection surface (
WORKFLOW_NAMEfromgithub.event.check_run.nameflows into the Claude prompt with--permission-mode bypassPermissions --allowedTools Bash) is pre-existing behavior, not introduced by this PR. Out of scope for this review, but worth tracking separately if the analyst is ever exposed to checks created by untrusted authors (forks are already filtered out, which mitigates this for now).
CI status
All 26 checks green or appropriately skipped: CodeQL, AgentShield, Lint, ShellCheck, bats, gh-aw-compile, Compile agentic workflows, Validate AW specs, unit-tests, SonarCloud (Quality Gate passed, 0 new issues), Secret scan (gitleaks), Agent Security Scan, PR Review Agent. Dependabot auto-merge and ecosystem-specific dependency-audit jobs correctly skipped (no manifest changes for npm/pnpm/pip/cargo/govulncheck).
Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 24b0e5f6fe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…crets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing.
- Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure.
- Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires.
- Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Collaborator
Author
|
No description provided. |
donpetry-bot
approved these changes
May 23, 2026
donpetry-bot
left a comment
donpetry-bot
left a comment
Contributor
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 32cf617e221a1c82546cb8a0e4e4a6d46c5d55bc
Review mode: triage-approved (single reviewer)
Summary
This PR extracts the CI Failure Analyst into a workflow_call reusable + thin caller stub, plus a copy-paste adoption template for other org repos. The architecture mirrors the established dev-lead-reusable.yml pattern. The analysis logic moves largely 1:1 (Claude Code invocation, idempotency check, prompt template) with sensible additions: an explicit anti-loop guard (!startsWith(check_run.name, 'CI Failure Analyst')), a non-fork PR filter at the resolution step, explicit workflow_call.secrets declaration in the reusable, and explicit per-secret pass-through (rather than secrets: inherit) from the callers.
This is the second cycle. The prior pass at 24b0e5f6 was dismissed because new commits followed addressing reviewer feedback — declaring CLAUDE_CODE_OAUTH_TOKEN under workflow_call: secrets:, adding issues: write to lock/template, pinning the reusable uses: ref, passing only the required secret explicitly, and refreshing the docs. Confirming triage approval.
Linked issue analysis
No linked issue. The PR body itself motivates the change (enable org-wide adoption without duplicating ~150 lines per repo). The architecture rationale (reusable vs. repository_dispatch) is well-justified — read-only diagnostic comment using the caller's own GITHUB_TOKEN, so no GH_PAT_WORKFLOWS relay is needed.
Findings
Both .github/workflows/ci-failure-analyst.lock.yml:31 and templates/ci-failure-analyst.yml:43 pin uses: to 69e9774818d45846f3a850ca13f31c7e9d6345cc # main. That SHA is the current main HEAD (the merge of #357), and the reusable workflow file ci-failure-analyst-reusable.yml does not exist at that SHA — it is introduced by this PR. Consequences after merge:
- The in-repo lock workflow will fire on the next
check_run: failure, try to resolve…/ci-failure-analyst-reusable.yml@69e9774, fail to find it, and the analyst will not run until the SHA is bumped. - Any other repo that adopts the template before the SHA is bumped will see the same failure.
Recommended fix: immediately after merging this PR, push a follow-up commit bumping the pinned SHA in both lock.yml and templates/ci-failure-analyst.yml to the merge commit of this PR. Alternatively, the lock could safely use @main (auto-tracking is fine for the in-repo caller since it lives in the same repo).
The prior dismissed review flagged this for the template only; it now also applies to the lock after commit a96d0e1 ("fix(lock): … pin SHA…"). Flagging more prominently here because the lock is the active workflow in this repo and will silently fail post-merge.
No other blocking issues.
Observations (informational):
- Anti-loop guard
!startsWith(github.event.check_run.name, 'CI Failure Analyst')correctly catches both"CI Failure Analyst"(lock) and"CI Failure Analyst — Reusable"(reusable) via the shared prefix. - Idempotency now intentionally skips (rather than updates) when a
<!-- ci-analyst sha=… -->comment already exists.tests/aw/ci-failure-analyst/scenarios.mdwas updated in the same PR to match the new behavior — good consistency, and the trade-off (only first failing check on a given SHA gets diagnosed) is documented in the scenario note. - The
shellcheck disable=SC2016comment at.github/workflows/ci-failure-analyst-reusable.yml:33is legitimate —$ENV.REPOinside a single-quoted jq filter is jq's environment-variable syntax, not a shell expansion. - Per-secret pass-through (
secrets: { CLAUDE_CODE_OAUTH_TOKEN: … }) instead ofsecrets: inheritis a deliberate minimization choice mentioned in commit messages — good security hygiene. - Prompt-injection surface (
WORKFLOW_NAMEfromgithub.event.check_run.nameflows into the Claude prompt running with--permission-mode bypassPermissions --allowedTools Bash) is pre-existing behavior, not introduced or worsened by this PR. The non-fork PR filter already mitigates the obvious attack vector; worth tracking separately if the analyst is ever exposed to checks from untrusted authors.
CI status
All 26 checks green or appropriately skipped: CodeQL (actions), AgentShield, CI Lint, ShellCheck, bats, validate-agent-profiles, gh-aw-compile, Compile agentic workflows, Validate AW specs, unit-tests, SonarCloud (Quality Gate passed, 0 new issues), Secret scan (gitleaks), Agent Security Scan, PR Review Agent. Dependabot auto-merge and ecosystem-specific dependency-audit jobs correctly skipped (no manifest changes for npm/pnpm/pip/cargo/govulncheck). CodeRabbit and SonarCloud status contexts also success.
Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 32cf617e22
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if: >- | ||
| github.event.check_run.conclusion == 'failure' && | ||
| !startsWith(github.event.check_run.name, 'CI Failure Analyst') | ||
| uses: petry-projects/.github-private/.github/workflows/ci-failure-analyst-reusable.yml@69e9774818d45846f3a850ca13f31c7e9d6345cc # main |
There was a problem hiding this comment.
The caller stub references petry-projects/.github-private/.github/workflows/ci-failure-analyst-reusable.yml@69e977..., but that commit predates the introduction of ci-failure-analyst-reusable.yml, so this workflow cannot be resolved at runtime and the analyst job will fail to start in this repo. Update the ref to a commit (or local same-repo path) that actually includes the reusable workflow file.
Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment.
Acknowledged — same root cause as the template comment. The SHA 69e9774 predated this PR and does not contain the reusable. PR #370 updates both ci-failure-analyst.lock.yml and the template to point to the merge commit 121bee881bd13715706d30230b2d5a8d2d78b0b1, which is the first main SHA that includes the reusable workflow file.
2 tasks
don-petry
added a commit
that referenced
this pull request
May 23, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
This was referenced May 23, 2026
don-petry
added a commit
that referenced
this pull request
Jun 7, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 7, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
don-petry
added a commit
that referenced
this pull request
Jun 7, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 7, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
don-petry
added a commit
that referenced
this pull request
Jun 8, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 8, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
don-petry
added a commit
that referenced
this pull request
Jun 12, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 12, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
don-petry
added a commit
that referenced
this pull request
Jun 14, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 14, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
don-petry
added a commit
that referenced
this pull request
Jun 18, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 18, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
This was referenced Jun 18, 2026
don-petry
added a commit
that referenced
this pull request
Jun 25, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 25, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
don-petry
added a commit
that referenced
this pull request
Jun 25, 2026
* feat(aw): add CI Failure Analyst reusable workflow * feat(aw): replace ci-failure-analyst.lock.yml with thin stub * feat(aw): add ci-failure-analyst template for org deployment * docs(aw): rewrite ci-failure-analyst.md for reusable workflow architecture * fix(tests): update duplicate-SHA scenario to match skip behavior The idempotency check in ci-failure-analyst-reusable.yml skips when a comment for the same SHA already exists (does not update in-place). Update the scenario spec to reflect the actual implementation and add a note explaining the intentional trade-off for multiple failing checks. * fix(templates): pin reusable workflow ref to commit SHA instead of @main Using a mutable branch ref (@main) exposes consumers to unexpected breaking changes. Pin to the current main HEAD SHA (69e9774) with a comment noting it tracks main for easy future updates. * fix(lint): suppress SC2016 false positive on jq $ENV.REPO filter $ENV.REPO inside the single-quoted jq filter string is jq built-in syntax for reading environment variables — shellcheck incorrectly flags it as an unexpanded shell variable. Add an inline disable comment to silence the false positive. * fix(reusable): declare CLAUDE_CODE_OAUTH_TOKEN under workflow_call secrets Declaring required secrets in the reusable's `on: workflow_call: secrets:` block (as dev-lead-reusable.yml does) gives callers an explicit validation error instead of a silent failure when the secret is missing. * fix(lock): add issues:write, pin SHA, pass secret explicitly - Add `issues: write` — PR comments are posted via the Issues comments API and require this permission alongside pull-requests: write. - Pin the reusable ref to the main HEAD SHA instead of @main to prevent unexpected behavior changes. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to minimize secret exposure. * fix(template): add issues:write and pass secret explicitly - Add `issues: write` — PR comments use the Issues comments API which requires this permission in addition to pull-requests: write. - Pass only CLAUDE_CODE_OAUTH_TOKEN explicitly rather than secrets: inherit to limit secret exposure to only what the reusable workflow requires. * fix(docs): update permissions table and architecture diagram - Add `issues: write` to permissions table — PR comments are posted via the Issues comments API and require this permission. - Correct `contents: read` description — it is for the runner environment, not a workflow_call requirement per se. - Update architecture diagram to show the pinned SHA ref instead of @main, matching the recommended deployment pattern from templates/.
don-petry
added a commit
that referenced
this pull request
Jun 25, 2026
* fix(template): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(lock): pin reusable ref to post-merge main SHA The SHA 69e9774 predated PR #367 and did not contain the reusable workflow. Update the pinned ref to the merge commit (121bee8) which is the first commit on main that includes ci-failure-analyst-reusable.yml. * fix(docs): update architecture SHA to post-merge main The SHA 69e9774 in the architecture diagram predated PR #367. Update to the merge commit (121bee8) which is the first main SHA that contains the reusable workflow.
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CI Failure Analyst currently only covers
.github-privateitself — each new org repo would need a full copy of the workflow logic to get diagnostic comments on CI failures. This PR extracts the logic into aworkflow_callreusable and provides a ~20-line stub template that any org repo can deploy.Problem
ci-failure-analyst.lock.ymlis a monolithic, hand-written workflow that lives and runs only in this repo. There is no mechanism to share it with other org repos without duplicating the entire 100+ line file — including the Claude invocation, idempotency logic, and prompt template.Solution
Split into two files following the established
dev-lead-reusable.ymlpattern:ci-failure-analyst-reusable.ymlci-failure-analyst.lock.ymltemplates/ci-failure-analyst.ymlWhy reusable workflow (not dispatch)
The existing cross-org pattern for dev-lead uses
repository_dispatchbecause dev-lead needs to push commits (requiresGH_PAT_WORKFLOWSwith write access to the target repo). CI Failure Analyst is read-only — it only posts a comment using the caller's ownGITHUB_TOKEN. This means:GH_PAT_WORKFLOWSneededsecrets: inheritDeploying to a new repo
Copy
templates/ci-failure-analyst.ymlto.github/workflows/ci-failure-analyst.ymlin the target repo. No other changes needed —CLAUDE_CODE_OAUTH_TOKENis already an org secret granted to all repos.Coexistence with dev-lead
Both can run in the same repo without conflict:
<!-- ci-analyst sha=... -->comment in < 2 min (diagnostic only)<!-- dev-lead sha=... -->comment + pushes a fix commit (5–10 min)Different idempotency markers, different comment authors, different run times. Having both gives developers a fast diagnosis while the fix is being prepared.
Changes
.github/workflows/ci-failure-analyst-reusable.yml— all logic;workflow_calltrigger.github/workflows/ci-failure-analyst.lock.yml— replaced with 20-line stubtemplates/ci-failure-analyst.yml— adoption template for other org reposdocs/aw/ci-failure-analyst.md— updated to reflect new architecture, removed stale gh-aw/ANTHROPIC_API_KEY referencesTest plan
ci-failure-analyst.lock.yml(this repo) fires oncheck_run: completed→ confirmed working pre-PRCI Failure Analystanti-loop guard prevents self-triggering