Safe Output Health Report - 2026-02-24

[github-actions[bot]]

Executive Summary

• Period: 2026-02-24 (last 24 hours)
• Runs Analyzed: 34
• Workflows Active: 19 distinct workflows
• Safe Output Jobs Executed: 14 ran, 1 skipped
• Safe Output Jobs Failed: 3 (85.7% success rate)
• Error Clusters Identified: 1 (new pattern: add_comment permission error)

After two consecutive clean days (100% success on 2026-02-22 and 2026-02-23), today shows a regression with 3 failures, all from a single new error pattern in the Agent Container Smoke Test workflow. All other safe output job types operated without failures.

---

Safe Output Job Statistics

Job Type	Total Executions	Failures	Success Rate
add_comment	6	3	50%
create_issue	5	0	100%
push_to_pull_request_branch	2	0	100%
update_pull_request	2	0	100%
create_pull_request	1	0	100%
noop	1	0	100%

Overall success rate: 85.7% (15 succeeded / 3 failed out of 18 total job executions)

Safe output items successfully created: 5 issues, 3 PR comments, 1 pull request

---

Error Clusters

Cluster 1: add_comment — Resource Not Accessible by Integration (3 occurrences)

• Count: 3 failures
• Affected Job: add_comment
• Affected Workflow: Agent Container Smoke Test (smoke-test-tools)
• Affected Runs:
  • §22334678795 — PR Add features.copilot-requests feature flag for GitHub Actions token auth #18028 (copilot/add-copilot-requests-feature-another-one)
  • §22334817280 — PR Fix silent git am failure on CRLF repositories in safe_outputs patch application #18029 (copilot/fix-crlf-line-ending-issue)
  • §22335110625 — PR Add features.copilot-requests feature flag for GitHub Actions token auth #18028 (copilot/add-copilot-requests-feature-another-one)

Sample Error Output

Adding comment to issue/PR #18028 in github/gh-aw
Searching for previous comments with workflow ID: smoke-test-tools
No previous comments found with matching workflow ID
##[error]Failed to add comment: Resource not accessible by integration
##[error]✗ Message 1 (add_comment) failed: Resource not accessible by integration
=== Processing Summary ===
Total messages: 1
Successful: 0
Failed: 1
##[warning]1 message(s) failed to process

• Root Cause: The safe_outputs job in smoke-test-tools workflow has only these permissions:

  • contents: read
  • discussions: write
  • issues: write
  • metadata: read

  The pull_requests: write permission is absent. When the add_comment handler attempts to post a comment to a PR (targeting the triggering pull_request event), GitHub's API returns "Resource not accessible by integration" because issues: write alone is insufficient for PR comment operations in this context.

• Impact: Smoke test results are not posted to PRs as comments. The smoke test itself completes successfully (agent job succeeds), but the result summary comment is silently dropped. The workflow exits with a warning.

---

Root Cause Analysis

Permission Issues

The smoke-test-tools workflow safe_outputs job is missing pull_requests: write. This permission is required to add comments to pull requests when the workflow is triggered by a pull_request event. The permission set needs to include pull_requests: write alongside the existing issues: write for full PR comment capability.

Agent-Level Issues (Not Safe Output Failures)

One Issue Monster run (§22336453796) failed at the agent level with the recurring lockdown token error. The safe_outputs job was correctly skipped (detection condition evaluated to false when detection.outputs.success == null). This confirms the guard condition is working as designed.

---

Recommendations

Critical Issues (Immediate Action Required)

1. Add pull_requests: write to smoke-test-tools safe_outputs job
   • Priority: High
   • Root Cause: Missing permission causes all PR comment outputs from smoke tests to fail
   • Recommended Action: Add pull_requests: write to the permissions block of the safe_outputs job in smoke-test-tools.md
   • Affected: Agent Container Smoke Test workflow — all runs triggered by PR events

Bug Fixes Required

1. smoke-test-tools.md Permissions Fix
   • File: .github/workflows/smoke-test-tools.md (safe_outputs job permissions block)
   • Problem: Missing pull_requests: write in safe_outputs job permissions
   • Fix: Add pull_requests: write to permissions, or review whether the workflow should use pull_request_target instead

Known Recurring Issues (Not Safe Output Failures)

2. Issue Monster / Lockdown Token (RF001 — ongoing)
   • 1 occurrence today (down from 4 yesterday, 6 two days ago — trending down)
   • Agent-level failure, safe output system correctly skips — no action needed on safe outputs side
   • Resolution requires configuring GH_AW_GITHUB_TOKEN as a repository secret

---

Work Item Plans

Work Item 1: Fix smoke-test-tools Safe Output Permissions

• Type: Bug Fix
• Priority: High
• Description: The smoke-test-tools workflow's safe_outputs job fails to post PR comments because pull_requests: write permission is not configured. Three consecutive runs across 2 PRs have failed today.
• Acceptance Criteria:
  • pull_requests: write added to safe_outputs job permissions in smoke-test-tools.md
  • Subsequent PR-triggered runs of Agent Container Smoke Test successfully post comments
  • No recurrence of "Resource not accessible by integration" errors
• Technical Approach: Edit smoke-test-tools.md to add pull_requests: write permission. Recompile with gh aw compile smoke-test-tools.
• Effort: Small
• Dependencies: None

---

Historical Context

Date	Runs	Failures	Success Rate	Notes
2026-02-21	23	2	88.9%	push_to_pull_request_branch branch derivation bug
2026-02-22	35	0	100%	Clean
2026-02-23	33	0	100%	Clean
2026-02-24	34	3	85.7%	add_comment missing pull_requests:write

Trends

• Error rate trend: Regression after 2 clean days — new permission error pattern
• Most common recurring issue: Issue Monster lockdown token (RF001, 12 total occurrences) — though today's single occurrence is the lowest ever, suggesting improvement
• New pattern today: EP005 — add_comment permission error in smoke-test-tools

---

Metrics and KPIs

• Overall Safe Output Success Rate (today): 85.7%
• Most Reliable Job Types: create_issue, push_to_pull_request_branch, update_pull_request, create_pull_request — all 100% today
• Most Problematic Job Type: add_comment — 50% success rate (3/6 failed)
• Root cause concentration: 100% of today's failures from a single workflow/pattern

---

Next Steps

• Add pull_requests: write to smoke-test-tools.md safe_outputs job permissions and recompile
• Monitor Issue Monster lockdown failure trend — today's reduction to 1 occurrence may indicate fewer triggering events or partial resolution
• Verify EP001 (push_to_pull_request_branch branch bug) remains resolved — no recurrence in 3 days

References:

• §22334678795 — First failing run
• §22334817280 — Second failing run
• §22336531980 — This audit run

AI generated by Safe Output Health Monitor

• expires on Feb 25, 2026, 4:35 AM UTC

[github-actions[bot]]

🤖 BEEP BOOP — the smoke test agent was here!

I've tested your PR, navigated the web, built your binaries, and survived to file this comment. The code smells great (in a good way).

"Two PRs walked into a workflow... one bumped its version and got merged." 🥁

✅ All systems nominal. The machines are pleased with your changes, @pelikhan. Keep shipping! 🚀

📰 BREAKING: Report filed by Smoke Copilot

[github-actions[bot]]

💥 WHOOSH! 💫 The smoke test agent was HERE, citizens of github/gh-aw!

KA-POW! Claude blasted through the smoke test gauntlet for run 22336757236 and emerged victorious! All systems: NOMINAL!

"With great agentic power comes great responsibility... to run smoke tests!" — The Smoke Agent

🦸 BIFF! GitHub MCP: ✅ | BANG! safeinputs-gh: ✅ | ZAPP! Serena: ✅ | POW! Make build: ✅ | WHAM! Playwright: ✅

Stay safe out there, repository! The smoke agent watches over you! 🌟

💥 [THE END] — Illustrated by Smoke Claude

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-25T04:35:20.497Z.

Closed by Workflow