TEST: Do not merge (testing spec review action) #15786

dmcilvaney · 2026-02-11T01:44:29Z

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

Summary

What does the PR accomplish, why was it needed?

Change Log

Change
Change
Change

Does this affect the toolchain?

YES/NO

Associated issues

#xxxx

Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-YYYY-XXXX

Test Methodology

Pipeline build id: xxxx

Add a github action which runs on pull request and adds inline annotations for changed specs based on upstream package standards. The check uses the Copilot CLI via a GitHub App user and requires the following secrets to be set in the repository: - COPILOT_TOKEN: The token for the GitHub App user. The action will run two parallel copies of the spec review agent with different models to review the changed specs and provide feedback. A third agent will then combine the feedback from both agents and create inline annotations on the pull request. Any annotation that is an error will cause the check to fail, while warnings and suggestions will be added as annotations without failing the check.

Add a custom agent compatible with both VSCode and GitHub CLI which reviews spec files.

Copilot

Pull request overview

Adds an automated “Spec Review” GitHub Actions workflow that runs a Copilot-powered review of changed .spec files, produces a structured JSON report, and posts annotations/comments back to the PR.

Changes:

Introduces .github/workflows/spec-review.yml plus supporting scripts to run multi-model spec reviews, validate JSON output, and publish PR comments/annotations/artifacts.
Adds a new go-md2man RPM spec file under base/comps/.
Updates ignore rules and a small comment tweak in azurelinux-release.spec.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
`test.txt`	Adds a one-line test file (appears to be a PR artifact).
`base/comps/go-md2man/go-md2man.spec`	New RPM spec for `go-md2man`.
`base/comps/azurelinux-release/azurelinux-release.spec`	Updates a TODO comment near EOL date macro.
`.gitignore`	Ignores Python caches and spec-review output directory/files.
`.github/workflows/spec-review.yml`	New workflow to run spec review, generate annotations, comment on PR, and upload artifacts.
`.github/workflows/scripts/spec_review_schema.py`	JSON schema validation + report comparison utility.
`.github/workflows/scripts/spec_review_multi.sh`	Orchestrates two review passes + synthesis pass.
`.github/workflows/scripts/spec_review.sh`	Core runner for Copilot agent, KB/log/report handling, validation loop.
`.github/workflows/scripts/requirements.txt`	Adds Python dependency for schema validation.
`.github/workflows/scripts/format_pr_comment.py`	Formats report into a PR comment with links and raw JSON section.
`.github/workflows/scripts/create_check_annotations.py`	Emits workflow-command annotations / Checks-style annotations from report.
`.github/workflows/scripts/_common.py`	Shared path helper for repo-relative linking.
`.github/workflows/scripts/README.md`	Local developer documentation for running the review scripts.
`.github/agents/spec-review.agent.md`	Defines the `spec-review` agent instructions for Copilot CLI.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-11T01:51:31Z

.github/workflows/spec-review.yml

+            SPEC_ARGS="$SPEC_ARGS --spec $spec"
+          done < "$SPEC_ARGS_FILE"
+          rm -f "$SPEC_ARGS_FILE"
+


After filtering suspicious spec paths, the script always sets skip=false even if all paths were filtered out, which will run the review with no --spec args (causing the scripts to scan all specs in the repo). Add a check for an empty validated list and set skip=true (or fail) to avoid unexpectedly expensive runs.

Suggested change

# If no valid specs remain after filtering, skip the review to avoid scanning all specs

if [[ -z "$SPEC_ARGS" ]]; then

echo "No valid .spec files to review after filtering suspicious paths"

echo "skip=true" >> "$GITHUB_OUTPUT"

exit 0

fi

Copilot · 2026-02-11T01:51:31Z

.github/workflows/spec-review.yml

+      - name: Install Copilot CLI
+        if: steps.changed-specs.outputs.skip != 'true'
+        run: |
+          npm i -g @github/copilot
+          copilot --version


npm i -g @github/copilot is unpinned, so the workflow can change behavior unexpectedly over time (and makes failures harder to reproduce). Consider installing a specific version (or using a lockfile/cached toolchain) so the CI environment is deterministic.

Copilot · 2026-02-11T01:51:31Z

.github/workflows/scripts/spec_review.sh

+    --allow-tool "shell(tail:*)" \
+    --allow-tool "shell(wc:*)" \
+    --allow-tool "shell(diff:*)" \
+    --allow-tool "shell(test:*)" \


The Copilot allowlist enables very broad shell access (e.g., shell(find:*), shell(cat:*), shell(diff:*)). If shell tooling isn’t constrained to the --add-dir roots, an injected prompt could use these to read arbitrary files (including runner files) and potentially exfiltrate secrets into logs/artifacts. Consider removing wildcard shell tools or tightening them to the minimal safe set needed for the agent to function.

Suggested change

--allow-tool "shell(test:*)" \

Copilot · 2026-02-11T01:51:32Z

base/comps/go-md2man/go-md2man.spec

+Vendor:         Microsoft Corporation
+Distribution:   Azure Linux


Vendor: and Distribution: are declared twice (once in the header and again after %define OUR_GOPATH). Duplicate spec tags can cause ambiguity/override behavior during build; please keep a single definition in the header.

Suggested change

Vendor: Microsoft Corporation

Distribution: Azure Linux

Copilot · 2026-02-11T01:51:32Z

base/comps/go-md2man/go-md2man.spec

+BuildRequires:  golang
+BuildRequires:  which
+# required packages on install
+Requires:       /bin/sh


Requires: /bin/sh looks unnecessary for a Go binary package (there are no scriptlets in this spec that would require a shell at runtime). Consider dropping it to avoid adding an artificial runtime dependency.

Suggested change

Requires: /bin/sh

Copilot · 2026-02-11T01:51:33Z

test.txt

@@ -0,0 +1 @@
+test


This file appears to be a PR/testing artifact and doesn’t look like it’s used by the repo (single line "test"). Consider removing it to avoid leaving stray files in the tree.

Suggested change

test

This file documents manual test procedures.

Copilot · 2026-02-11T01:51:33Z

base/comps/azurelinux-release/azurelinux-release.spec

+# TODO(azl): review -- set to a future date for now, fix before release
 %define eol_date 1900-01-01


The TODO says the EOL date was set to a future date, but %define eol_date 1900-01-01 is in the past. Please update the comment or set eol_date to the intended value to avoid misleading release metadata.

Suggested change

# TODO(azl): review -- set to a future date for now, fix before release

%define eol_date 1900-01-01

# TODO(azl): review -- temporary far-future EOL; set real date before release

%define eol_date 2099-12-31

Copilot · 2026-02-11T01:51:33Z

base/comps/go-md2man/go-md2man.spec

+URL:            https://github.com/cpuguy83/go-md2man
+Source0:        https://github.com/cpuguy83/go-md2man/archive/v%{version}.tar.gz#/go-md2man-%{version}.tar.gz
+BuildRequires:  golang
+BuildRequires:  which


BuildRequires: which doesn’t appear to be used anywhere in this spec. Consider removing it to avoid unnecessary build dependencies.

Suggested change

BuildRequires: which

Copilot · 2026-02-11T01:51:33Z

base/comps/go-md2man/go-md2man.spec

+export CGO_ENABLED=0
+export GO111MODULE=on
+
+cd %{_topdir}/BUILD/%{name}-%{version}/go-md2man-%{version}


The build step hard-codes %{_topdir}/BUILD/... for the working directory. This is less portable than using the standard rpm macros/working directory (e.g., %{_builddir} / staying in the directory created by %setup) and can break in nonstandard buildroots.

Suggested change

cd %{_topdir}/BUILD/%{name}-%{version}/go-md2man-%{version}

cd %{_builddir}/%{name}-%{version}/go-md2man-%{version}

Copilot · 2026-02-11T01:51:34Z

.github/workflows/spec-review.yml

+      - name: Test gh auth
+        env:
+          GH_TOKEN: ${{ secrets.COPILOT_TOKEN }}
+        run: |
+          gh auth status


This workflow uses secrets.COPILOT_TOKEN on the pull_request event. For PRs from forks, GitHub won’t provide repository secrets, so this step (and the job) will fail and block CI. Consider adding a guard to skip the job when the secret is unavailable / when github.event.pull_request.head.repo.fork is true, and emit a neutral message instead.

dmcilvaney added 7 commits February 6, 2026 17:16

Remove emojis

1c1d57a

copilot feedback

9cdd0d3

ci(pr): Add spec review agent

8aaafe2

Add a custom agent compatible with both VSCode and GitHub CLI which reviews spec files.

Apply suggestions from code review

9c3db3b

test commit

6ece26a

better test

8e57e4e

Copilot AI review requested due to automatic review settings February 11, 2026 01:44

Copilot started reviewing on behalf of dmcilvaney February 11, 2026 01:45 View session

dmcilvaney closed this Feb 11, 2026

Copilot AI reviewed Feb 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TEST: Do not merge (testing spec review action) #15786

TEST: Do not merge (testing spec review action) #15786

Uh oh!

dmcilvaney commented Feb 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

+          # If no valid specs remain after filtering, skip the review to avoid scanning all specs
+          if [[ -z "$SPEC_ARGS" ]]; then
+            echo "No valid .spec files to review after filtering suspicious paths"
+            echo "skip=true" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi

		# TODO(azl): review -- set to a future date for now, fix before release
		%define eol_date 1900-01-01

	cd %{_topdir}/BUILD/%{name}-%{version}/go-md2man-%{version}
	cd %{_builddir}/%{name}-%{version}/go-md2man-%{version}

TEST: Do not merge (testing spec review action) #15786

TEST: Do not merge (testing spec review action) #15786

Uh oh!

Conversation

dmcilvaney commented Feb 11, 2026

Merge Checklist

Summary

Change Log

Does this affect the toolchain?

Associated issues

Links to CVEs

Test Methodology

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant