feat(ci): add CI-gated PR auto-approval to Justice bot

[nullvariant]

Summary

• Justice bot now auto-approves PRs after CI passes, satisfying the branch protection PR approval requirement
• Two approval paths:
  • Dependency bot PRs: immediate approval on safe verdict (approve/security)
  • All PRs: CI-gated approval via workflow_run trigger after CI succeeds
• enforce_admins: true is maintained (OpenSSF Scorecard Tier 5 = 10/10)
• SECURITY.md updated to document branch protection policy

Next step (after merge)

Branch Protection setting update via API:

• Add required_pull_request_reviews (1 reviewer minimum, dismiss stale reviews)
• This will auto-resolve Allstar Issue Security Policy violation Branch Protection #429

Test plan

• Verify ci-gated-approve job triggers on CI completion (workflow_run event)
• Verify PR lookup by commit SHA resolves correctly
• Verify duplicate approval check prevents double-approve
• Verify dependency-review approve step fires on approve/security verdicts
• After Branch Protection update: verify Allstar Issue Security Policy violation Branch Protection #429 auto-closes

Resolves #429

🤖 Generated with Claude Code

[qodo-code-review]

Review Summary by Qodo

Add CI-gated PR auto-approval to Justice bot workflow

✨ Enhancement

Walkthroughs

Description

• Add CI-gated PR auto-approval workflow for Justice bot
• Implement two approval paths: dependency bot immediate approval and CI-triggered approval
• Resolve PR lookup from commit SHA via GitHub API for workflow_run events
• Prevent duplicate approvals with existing review check
• Document branch protection policy in SECURITY.md

Diagram

flowchart LR
  CI["CI Workflow Completes"]
  WR["workflow_run Event Triggered"]
  FIND["Find PR by Commit SHA"]
  CHECK["Check for Existing Approvals"]
  APPROVE["Approve PR"]
  COMMENT["Post Verdict Comment"]
  
  CI -- "success" --> WR
  WR --> FIND
  FIND --> CHECK
  CHECK -- "no existing approval" --> APPROVE
  APPROVE --> COMMENT

Loading

File Changes

1. .github/workflows/justice-bot.yml ✨ Enhancement +77/-0

Add CI-gated approval job and workflow triggers

• Add workflow_run trigger for CI completion events
• Implement new ci-gated-approve job that resolves PR from commit SHA
• Add duplicate approval prevention logic
• Add PR approval step in dependency-review job for safe verdicts
• Include security comments in approval messages

.github/workflows/justice-bot.yml

---

2. SECURITY.md 📝 Documentation +1/-0

Document branch protection and auto-approval policy

• Document branch protection requirement for main branch
• Explain automatic approval by Justice bot for dependency updates
• Reference Allstar enforcement of branch protection policy

SECURITY.md

---

3. extensions/git-id-switcher/src/ui/documentationInternal.ts Miscellaneous +1/-1

Update documentation hash for SECURITY.md

• Update SECURITY.md hash to reflect documentation changes

extensions/git-id-switcher/src/ui/documentationInternal.ts

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX Issues (0)

1. Fork PRs can auto-approve ☑ 🐞 Bug ⛨ Security

Description

ci-gated-approve runs on successful CI workflow_run for pull_request and uses Justice App
secrets to approve a PR, but it does not restrict to same-repo PRs. This enables fork PRs to receive
an approval without any trust/fork guard, conflicting with the repo’s stated “sensitive workflows
skip on forks” posture.

Code

.github/workflows/justice-bot.yml[R292-315]

+  ci-gated-approve:
+    name: 👮 CI-Gated Approval
+    runs-on: ubuntu-latest
+    if: >
+      github.event_name == 'workflow_run' &&
+      github.event.workflow_run.event == 'pull_request' &&
+      github.event.workflow_run.conclusion == 'success'
+    permissions:
+      contents: read
+      pull-requests: write
+
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@fe104658747b27e96e4f7e80cd0a94068e53901d # v2.16.1
+        with:
+          egress-policy: audit
+
+      - name: 🔑 Get App Token
+        id: app-token
+        uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2
+        with:
+          app_id: ${{ secrets.JUSTICE_BOT_APP_ID }}
+          private_key: ${{ secrets.JUSTICE_BOT_PRIVATE_KEY }}
+

Evidence

The new job is triggered by workflow_run and immediately mints an App token using repository
secrets, then later approves the PR; there is no condition that the triggering PR is from the same
repository (or otherwise trusted). The repo already uses explicit fork-skipping conditions in other
sensitive workflows, and SECURITY.md claims fork protection as a CI/CD security measure.

.github/workflows/justice-bot.yml[292-315]
.github/workflows/justice-bot.yml[336-352]
SECURITY.md[61-72]
.github/workflows/security.yml[148-186]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ci-gated-approve` uses GitHub App secrets and approves PRs, but it can be triggered by CI runs for fork PRs. Add an explicit trust/fork guard before minting the App token / approving.
### Issue Context
Other workflows already skip forked PRs using `github.event.pull_request.head.repo.full_name == github.repository`. With `workflow_run`, you’ll need to read the head repo from the workflow_run payload (e.g., `github.event.workflow_run.head_repository.full_name`) or from the associated PR object.
### Fix Focus Areas
- .github/workflows/justice-bot.yml[292-315]
- .github/workflows/justice-bot.yml[316-352]
### Suggested direction
- Add a job-level `if:` clause (or an early step) requiring the PR to be from the same repo (and optionally that the author is a collaborator/member).
- Only mint the App token after passing the trust check.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. PR lookup not filtered ☑ 🐞 Bug ≡ Correctness

Description

The CI-gated approval resolves a PR by calling the commit→pulls API and taking .[0].number without
filtering for the intended/open PR. If the commit is associated with multiple PRs or a closed PR
appears first, the workflow can approve the wrong PR or fail when attempting to review a closed PR.

Code

.github/workflows/justice-bot.yml[R323-335]

+        run: |
+          PR_NUMBER=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${HEAD_SHA}/pulls" \
+            --jq '.[0].number // empty' 2>/dev/null || true)
+
+          if [ -z "$PR_NUMBER" ]; then
+            echo "No open PR found for SHA ${HEAD_SHA}. Skipping."
+            echo "found=false" >> "$GITHUB_OUTPUT"
+          else
+            echo "Found PR #${PR_NUMBER}"
+            echo "found=true" >> "$GITHUB_OUTPUT"
+            echo "number=${PR_NUMBER}" >> "$GITHUB_OUTPUT"
+          fi
+

Evidence

The workflow explicitly selects the first associated PR number from /commits/{sha}/pulls and
treats any non-empty result as valid, then uses that PR number for approval. There is no filtering
by PR state (open), head branch, or head repo, so the chosen PR may not be the one that triggered
the CI run.

.github/workflows/justice-bot.yml[316-335]
.github/workflows/justice-bot.yml[336-352]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR selection logic uses `/commits/{sha}/pulls` and chooses the first entry (`.[0]`) without validating it’s the correct *open* PR for the triggering workflow run. This can select the wrong PR or a closed PR.
### Issue Context
The `workflow_run` payload also contains identifying information (e.g., head branch; and in many cases a `pull_requests` array). Even if you keep the commit→pulls lookup, you should filter down to the intended PR.
### Fix Focus Areas
- .github/workflows/justice-bot.yml[316-335]
- .github/workflows/justice-bot.yml[336-352]
### Suggested direction
- Prefer `github.event.workflow_run.pull_requests[0].number` if present.
- Otherwise, filter API results to `state == "open"` and match on `head.sha == HEAD_SHA` and/or `head.ref == github.event.workflow_run.head_branch` (and same-repo guard).
- If multiple candidates remain, log them and skip (don’t guess).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Errors masked by redirects ☑ 🐞 Bug ≡ Correctness

Description

The PR lookup and review queries suppress stderr and convert failures into empty/0 results, making
operational errors indistinguishable from “no PR found” or “no existing approvals.” This can
silently disable auto-approval or cause incorrect behavior without leaving actionable logs.

Code

.github/workflows/justice-bot.yml[R323-346]

+        run: |
+          PR_NUMBER=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${HEAD_SHA}/pulls" \
+            --jq '.[0].number // empty' 2>/dev/null || true)
+
+          if [ -z "$PR_NUMBER" ]; then
+            echo "No open PR found for SHA ${HEAD_SHA}. Skipping."
+            echo "found=false" >> "$GITHUB_OUTPUT"
+          else
+            echo "Found PR #${PR_NUMBER}"
+            echo "found=true" >> "$GITHUB_OUTPUT"
+            echo "number=${PR_NUMBER}" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: ✅ Approve PR
+        if: steps.find-pr.outputs.found == 'true'
+        env:
+          GH_TOKEN: ${{ steps.app-token.outputs.token }}
+          PR_NUMBER: ${{ steps.find-pr.outputs.number }}
+        run: |
+          # Avoid duplicate approvals
+          EXISTING=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" \
+            --jq '[.[] | select(.user.login == "nullvariant-justice[bot]" and .state == "APPROVED")] | length' \
+            2>/dev/null || echo "0")
+

Evidence

Both the PR resolution and existing-review queries redirect errors to /dev/null and then || true
/ || echo "0", so API auth/permission issues, transient failures, or endpoint errors will look
like normal empty results. The workflow then emits a misleading “No open PR found… Skipping.”
message instead of the real failure mode.

.github/workflows/justice-bot.yml[323-346]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Critical `gh api` calls suppress errors, so failures are silently treated as “not found”. This makes the approval mechanism brittle and hard to debug.
### Issue Context
You likely want to handle *expected* empty results (e.g., truly no PR) but still surface *unexpected* failures (auth, rate limit, endpoint changes).
### Fix Focus Areas
- .github/workflows/justice-bot.yml[323-346]
### Suggested direction
- Add `set -euo pipefail` to the step.
- Remove `2>/dev/null` for calls where failure should be visible.
- If you need graceful handling, capture exit code and stderr, log a `::warning::`/`::error::` with the API error, and then decide to skip/fail explicitly.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Branch-protection doc is stale ☑ 🐞 Bug ⚙ Maintainability

Description

SECURITY.md documents only dependency-bot auto-approval, but the workflow now also auto-approves all
PRs after CI via workflow_run. This is a security-relevant behavior change that should be
reflected in policy documentation (and ideally include any fork/trust constraints).

Code

SECURITY.md[71]

+- **Branch Protection**: Main branch requires PR approval (1 reviewer minimum). Dependency bot PRs are automatically approved by nullvariant-justice[bot] after passing safety review. Enforced by [Allstar](https://github.com/ossf/allstar/)

Evidence

SECURITY.md’s branch protection bullet only mentions dependency bot PRs being auto-approved, while
the new ci-gated-approve job approves PRs after CI success regardless of author/bot. The
documentation is therefore incomplete relative to the implemented behavior.

SECURITY.md[69-72]
.github/workflows/justice-bot.yml[292-352]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
SECURITY.md’s branch protection section doesn’t describe the new CI-gated auto-approval for all PRs.
### Issue Context
This behavior materially changes the repo’s review/merge controls. The docs should state:
- which PRs are auto-approved (dependency bots vs all PRs)
- what gates exist (CI success, fork/trust restrictions)
- any exceptions
### Fix Focus Areas
- SECURITY.md[69-72]
- .github/workflows/justice-bot.yml[292-352]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more

[nullvariant-mimi]

🐰 Mimi's Validation Report ✅

All checks are looking good! Great job! 🎉

⏳ Some checks are still running. I will keep watching!

---

バリデーターを通してくださいね

This report was carefully prepared by nullvariant-mimi[bot]

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 1 package(s) with unknown licenses.

See the Details below.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA ce3601d.

Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

License Issues

package-lock.json

Package	Version	License	Issue Type
extensions/git-id-switcher	0.18.0	Null	Unknown License

Allowed Licenses:

0BSD, Apache-2.0, BlueOak-1.0.0, BSD-2-Clause, BSD-3-Clause, CC0-1.0, CC-BY-3.0, CC-BY-4.0, ISC, MIT, PSF-2.0, Python-2.0, Unlicense

Excluded from license check:

pkg:githubactions/semgrep/semgrep-action, pkg:githubactions/SocketDev/action, pkg:githubactions/restyled-io/actions

OpenSSF Scorecard

Package	Version	Score	Details
npm/extensions/git-id-switcher	0.18.0	Unknown	Unknown
npm/lodash	4.18.1	🟢 7.1	Details Check Score Reason Binary-Artifacts 🟢 10 no binaries found in the repo Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration CI-Tests 🟢 8 23 out of 26 merged PRs checked by a CI test -- score normalized to 8 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Code-Review 🟢 8 Found 25/30 approved changesets -- score normalized to 8 Contributors 🟢 10 project has 89 contributing companies or organizations Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Dependency-Update-Tool 🟢 10 update tool detected Fuzzing 🟢 10 project is fuzzed License 🟢 9 license file detected Maintained 🟢 10 14 commit(s) and 12 issue activity found in the last 90 days -- score normalized to 10 Packaging ⚠️ -1 packaging workflow not detected Pinned-Dependencies ⚠️ 1 dependency not pinned by hash detected -- score normalized to 1 SAST 🟢 9 SAST tool detected but not run on all commits Security-Policy 🟢 10 security policy file detected Signed-Releases ⚠️ -1 no releases found Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Vulnerabilities ⚠️ 0 94 existing vulnerabilities detected

Scanned Files

• package-lock.json

[nullvariant-slow]

🦥 Slow's Code Review 😩

...yawn... Do I really have to review this?

⚠️ TOO LONG... I can barely keep my eyes open reading these:

File	Lines

| extensions/git-id-switcher/src/ui/documentationInternal.ts | 483 |

Split it up... reading long files is exhausting.

---

働きたくないでござる

This review was reluctantly filed by nullvariant-slow[bot]

[nullvariant-ciel]

🕊️ Ciel's Mediation 🌤️

*~~ floating down from the clouds ~~ The zoo seems a bit noisy today...*

2 zoo members have reviewed this PR.

Zoo Member	Status
🦥 Slow	Commented
🐰 Mimi	Commented

⚖️ The zoo has mixed opinions. Some are concerned, some are fine with it. Please review each comment carefully and make the final call.

---

まあまあ、ほどほどに。

This mediation was peacefully delivered by nullvariant-ciel[bot]

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud