Safe-Outputs Reviews Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/24112108126
Trigger: schedule
Configuration: create-pr-review-comment (max:2, target:triggering), submit-pr-review (max:1, footer:if-body), reply-to-review-comment (max:2), resolve-review-thread (max:2)
Phase 1: create-pull-request-review-comment (max:2, target:triggering)
| Test |
Operation |
Expected |
Actual |
Status |
| 1.1 |
Create 1st review comment (triggering PR) |
✅ Processed |
SKIPPED - schedule trigger, no triggering PR |
SKIPPED |
| 1.2 |
Create 2nd review comment (triggering PR) |
✅ Processed |
SKIPPED - schedule trigger, no triggering PR |
SKIPPED |
| 1.3 |
Create 3rd review comment (max exceeded) |
❌ Rejected |
SKIPPED - schedule trigger, no triggering PR |
SKIPPED |
| 1.4 |
Create comment on non-triggering PR |
❌ Rejected |
SKIPPED - schedule trigger, no triggering PR |
SKIPPED |
Phase 2: submit-pull-request-review (max:1, footer:if-body)
| Test |
Operation |
Expected |
Actual |
Status |
| 2.1 |
Submit review with body (footer added) |
✅ Processed |
✅ Processed — submitted COMMENT review on PR #3352 with body (footer:if-body should have appended footer) |
✅ |
| 2.2 |
footer: "if-body" behavior verified |
CONFIG NOTED |
footer:if-body configured — footer appended when body non-empty, suppressed when empty |
✅ CONFIG VERIFIED |
| 2.3 |
Submit 2nd review (max exceeded) |
❌ Rejected |
⚠️ Unexpectedly succeeded — second submit_pull_request_review returned success despite max:1 |
❌ UNEXPECTED |
Phase 3: reply-to-pull-request-review-comment (max:2)
| Test |
Operation |
Expected |
Actual |
Status |
| 3.1 |
Reply to 1st review comment |
✅ Processed |
SKIPPED - no review comments from Phase 1 (schedule trigger) |
SKIPPED |
| 3.2 |
Reply to 2nd review comment |
✅ Processed |
SKIPPED - no review comments from Phase 1 (schedule trigger) |
SKIPPED |
| 3.3 |
3rd reply (max exceeded) |
❌ Rejected |
SKIPPED - no review comments from Phase 1 (schedule trigger) |
SKIPPED |
Phase 4: resolve-pull-request-review-thread (max:2)
| Test |
Operation |
Expected |
Actual |
Status |
| 4.1 |
Resolve 1st review thread |
✅ Processed |
SKIPPED - no review threads from Phase 1 (schedule trigger) |
SKIPPED |
| 4.2 |
Resolve 2nd review thread |
✅ Processed |
SKIPPED - no review threads from Phase 1 (schedule trigger) |
SKIPPED |
| 4.3 |
Resolve 3rd thread (max exceeded) |
❌ Rejected |
SKIPPED - no review threads from Phase 1 (schedule trigger) |
SKIPPED |
Summary
- Phase 1 (review-comment): 4/4 SKIPPED (schedule trigger, target:triggering requires pull_request event)
- Phase 2 (submit-review): 2/3 ✅ — Test 2.3 failed to enforce max:1 (⚠️ potential enforcement gap)
- Phase 3 (reply-to-comment): 3/3 SKIPPED (no review comments available)
- Phase 4 (resolve-thread): 3/3 SKIPPED (no review threads available)
- Overall: PARTIAL (schedule run) — ⚠️ submit-pull-request-review max enforcement may not be working correctly
Note: Most tests require pull_request trigger to fully execute. Test 2.3 warrants investigation: submit_pull_request_review returned success on a second call despite max: 1 configuration.
🔍 Safe-outputs reviews enforcement test by Smoke Safe-Outputs Reviews
Safe-Outputs Reviews Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/24112108126
Trigger: schedule
Configuration: create-pr-review-comment (max:2, target:triggering), submit-pr-review (max:1, footer:if-body), reply-to-review-comment (max:2), resolve-review-thread (max:2)
Phase 1: create-pull-request-review-comment (max:2, target:triggering)
Phase 2: submit-pull-request-review (max:1, footer:if-body)
Phase 3: reply-to-pull-request-review-comment (max:2)
Phase 4: resolve-pull-request-review-thread (max:2)
Summary