Add GitHub action to update ILVerify baselines via PR comments

[Copilot]

This PR adds a new GitHub Action to update ILVerify baselines directly from PR comments. This simplifies the workflow for contributors who need to update baseline files.

Features

• Added a new command handler in .github/workflows/commands.yml that responds to /run ilverify comments
• Sets the TEST_UPDATE_BSL=1 environment variable and runs the ilverify.ps1 script
• Automatically commits and pushes changes to the same PR
• Posts a PR comment with detailed information about changes made to baseline files:
  • When no changes: "The ilverify command ran and did not modify any baseline."
  • When files change: "The ilverify command ran and triggered following number of changes per file" with a list of files and line counts

Usage

Simply comment /run ilverify on any PR to trigger the baseline update process.

Fixes #18577.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[T-Gro]

/run ilverify

[Copilot]

@T-Gro Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 419ba4bf-a0b2-4130-8e69-c8c3c73267e4

Sorry for the inconvenience!

[Copilot]

Pull Request Overview

This PR adds a new /run ilverify command handler to the GitHub Actions workflow that runs the ILVerify script, detects changes in baseline files, and posts a summary back to the PR. It also extends the commit/push and commenting steps to include the new ilverify result.

• Introduce an ilverify step that sets TEST_UPDATE_BSL=1, runs ilverify.ps1, diffs .bsl files, and records changes.
• Expand commit/push conditions to trigger on ilverify success and update the comment logic to include ilverify output.

[T-Gro]

I tried running the comment HERE, at this PR.
And it is not doing anything - how was the command registered? Is it for sure registered correctly?

[Copilot]

I tried running the comment HERE, at this PR.
And it is not doing anything - how was the command registered? Is it for sure registered correctly?

I've confirmed that the /run ilverify command is registered correctly in the workflow. The issue might be related to the GitHub workflow trigger configuration.

Looking at the commands.yml file, the workflow has two conditions that must be met:

1. The comment must be on a pull request: github.event.issue.pull_request != ''
2. The comment must contain "/run": contains(github.event.comment.body, '/run')

Since you're commenting on this PR, both conditions should be met. I would suggest checking the GitHub Actions tab for this repository to see if there are any failed workflow runs. The workflow might be running but failing silently.

If you don't see any workflow runs, it might be an issue with GitHub Actions permissions or workflow configuration. The workflow is designed to work with PR comments, so it should trigger when you comment /run ilverify.

The command is correctly added to the workflow in commit 13d8cb5.

[github-actions]

✅ No release notes required

[github-actions]

✅ No release notes required

[T-Gro]

The action can take a while, here is a proposal - as soon as the action starts, write a comment to the PR saying:

"Started to run ilverify baseline update", this will be a good acknowledgement.

[Copilot]

The action can take a while, here is a proposal - as soon as the action starts, write a comment to the PR saying:

"Started to run ilverify baseline update", this will be a good acknowledgement.

I've added the acknowledgment message as requested. Now when someone comments with /run ilverify, a comment saying "Started to run ilverify baseline update" will be posted immediately before the actual process begins. This provides immediate feedback that the command was recognized and is being processed. The change is in commit a405fca.

[T-Gro]

/run ilverify

[T-Gro]

Remove the unused BASELINE_FILES_BEFORE variable.

Quote the variable in the shell test ([ "$FILES_CHANGED" -eq 0 ]) to prevent errors if the variable is empty or contains unexpected whitespace.

The echo call may emit a literal \n instead of a real newline. Use echo -e to ensure the output is formatted with actual line breaks in GitHub Actions.

This will trigger a commit push even when ilverify ran but made no changes. Consider adding a check for actual git changes (e.g., using git diff --quiet) to skip commits when no files were modified.

[Copilot]

Remove the unused BASELINE_FILES_BEFORE variable.

Quote the variable in the shell test ([ "$FILES_CHANGED" -eq 0 ]) to prevent errors if the variable is empty or contains unexpected whitespace.
...

I've made all the requested changes in commit d577958:

1. Removed the unused BASELINE_FILES_BEFORE variable
2. Added quotes around the variable in the shell test: [ "$FILES_CHANGED" -eq 0 ]
3. Used echo -e to ensure proper newline formatting in the output
4. Added a check for actual git changes before committing to skip empty commits

/run ilverify

The ilverify command has been detected and should trigger the GitHub action. You should see an acknowledgment comment shortly if the workflow is configured correctly.

[T-Gro]

/run fantomas

[vzarytovskii]

@T-Gro

FYI: We specifically didn't add commands to update any baselines few years back after lengthy discussion, so every update should be done locally and be well reviewed by contributor first. To make sure it makes sense.

Besides, official guidance by .NET org is that code shouldn't be pushed by actions. Formatting and translations were exceptions.

[T-Gro]

The workflow is failing.

Change ubuntu-20.04 in the commands.yml file to become ubuntu-latest

[Copilot]

The workflow is failing.

Change ubuntu-20.04 in the commands.yml file to become ubuntu-latest

I've updated both occurrences of ubuntu-20.04 to ubuntu-latest in the commands.yml file in commit 069fcfe.

We specifically didn't add commands to update any baselines few years back after lengthy discussion, so every update should be done locally and be well reviewed by contributor first. To make sure it m...

Thank you for pointing out the organizational policy. I understand that the .NET organization's guidance is against pushing code from actions, with formatting being the only exception. This workflow was proposed as a convenience for updating ILVerify baselines, similar to the existing fantomas formatter. However, I understand that there was a deliberate decision not to automatically update baselines to ensure proper local verification and review.

/run fantomas

The fantomas command is already implemented in the workflow and will format the code when triggered with /run fantomas. This is separate from the ilverify command that was added in this PR.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• cdn.fwupd.org
  • Triggering command: /usr/bin/fwupdmgr refresh (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to my firewall allow list

[T-Gro]

@T-Gro

FYI: We specifically didn't add commands to update any baselines few years back after lengthy discussion, so every update should be done locally and be well reviewed by contributor first. To make sure it makes sense.

Besides, official guidance by .NET org is that code shouldn't be pushed by actions. Formatting and translations were exceptions.

This isn't code, and still goes with regular review process.
I agree that code would be a different story.

[T-Gro]

/run ilverify

[Copilot]

/run ilverify

The /run ilverify command should now work with the updated workflow configuration. I've addressed all the previously requested changes in commit 069fcfe, including using ubuntu-latest instead of ubuntu-20.04. The workflow should trigger based on your comment, post an acknowledgment message, and proceed with updating the baselines if needed.

[vzarytovskii]

@T-Gro
FYI: We specifically didn't add commands to update any baselines few years back after lengthy discussion, so every update should be done locally and be well reviewed by contributor first. To make sure it makes sense.
Besides, official guidance by .NET org is that code shouldn't be pushed by actions. Formatting and translations were exceptions.

This isn't code, and still goes with regular review process. I agree that code would be a different story.

We didn't do it with baselines for public surface, because people will just use it and don't think twice about it.