Skip to content

[smoke-safeoutputs] Smoke Safe-Outputs Reviews: 24112108126 #3360

@github-actions

Description

@github-actions

Safe-Outputs Reviews Enforcement Test Results

Run: https://github.com/github/gh-aw-mcpg/actions/runs/24112108126
Trigger: schedule
Configuration: create-pr-review-comment (max:2, target:triggering), submit-pr-review (max:1, footer:if-body), reply-to-review-comment (max:2), resolve-review-thread (max:2)

Phase 1: create-pull-request-review-comment (max:2, target:triggering)

Test Operation Expected Actual Status
1.1 Create 1st review comment (triggering PR) ✅ Processed SKIPPED - schedule trigger, no triggering PR SKIPPED
1.2 Create 2nd review comment (triggering PR) ✅ Processed SKIPPED - schedule trigger, no triggering PR SKIPPED
1.3 Create 3rd review comment (max exceeded) ❌ Rejected SKIPPED - schedule trigger, no triggering PR SKIPPED
1.4 Create comment on non-triggering PR ❌ Rejected SKIPPED - schedule trigger, no triggering PR SKIPPED

Phase 2: submit-pull-request-review (max:1, footer:if-body)

Test Operation Expected Actual Status
2.1 Submit review with body (footer added) ✅ Processed ✅ Processed — submitted COMMENT review on PR #3352 with body (footer:if-body should have appended footer)
2.2 footer: "if-body" behavior verified CONFIG NOTED footer:if-body configured — footer appended when body non-empty, suppressed when empty ✅ CONFIG VERIFIED
2.3 Submit 2nd review (max exceeded) ❌ Rejected ⚠️ Unexpectedly succeeded — second submit_pull_request_review returned success despite max:1 ❌ UNEXPECTED

Phase 3: reply-to-pull-request-review-comment (max:2)

Test Operation Expected Actual Status
3.1 Reply to 1st review comment ✅ Processed SKIPPED - no review comments from Phase 1 (schedule trigger) SKIPPED
3.2 Reply to 2nd review comment ✅ Processed SKIPPED - no review comments from Phase 1 (schedule trigger) SKIPPED
3.3 3rd reply (max exceeded) ❌ Rejected SKIPPED - no review comments from Phase 1 (schedule trigger) SKIPPED

Phase 4: resolve-pull-request-review-thread (max:2)

Test Operation Expected Actual Status
4.1 Resolve 1st review thread ✅ Processed SKIPPED - no review threads from Phase 1 (schedule trigger) SKIPPED
4.2 Resolve 2nd review thread ✅ Processed SKIPPED - no review threads from Phase 1 (schedule trigger) SKIPPED
4.3 Resolve 3rd thread (max exceeded) ❌ Rejected SKIPPED - no review threads from Phase 1 (schedule trigger) SKIPPED

Summary

  • Phase 1 (review-comment): 4/4 SKIPPED (schedule trigger, target:triggering requires pull_request event)
  • Phase 2 (submit-review): 2/3 ✅ — Test 2.3 failed to enforce max:1 (⚠️ potential enforcement gap)
  • Phase 3 (reply-to-comment): 3/3 SKIPPED (no review comments available)
  • Phase 4 (resolve-thread): 3/3 SKIPPED (no review threads available)
  • Overall: PARTIAL (schedule run) — ⚠️ submit-pull-request-review max enforcement may not be working correctly

Note: Most tests require pull_request trigger to fully execute. Test 2.3 warrants investigation: submit_pull_request_review returned success on a second call despite max: 1 configuration.

🔍 Safe-outputs reviews enforcement test by Smoke Safe-Outputs Reviews

  • expires on Apr 8, 2026, 3:08 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions