Component
Scripts
Bug Description
Several eval scripts under scripts/evals write GitHub Actions workflow commands (::error / ::warning) by interpolating file paths and messages directly into the command string, without the percent-encoding GitHub's runner expects. The same applies to two moderation warnings that echo file paths on plain log lines.
When a path or message contains characters that are special in workflow commands (%, :, ,, or a line break), the annotation is truncated, attached to the wrong file or line, or split into additional log lines that the runner may interpret. This came up as an out-of-scope observation while reviewing #3139.
Affected emitters: Modules/ModerationRunner.psm1, Invoke-VallyEvals.ps1, Test-EvalSpec.ps1, Test-EvalSpecText.ps1, Test-StimulusPresence.ps1, Test-VallyTestSafety.ps1, and Invoke-ArtifactModeration.ps1.
The affected jobs run without secrets and with a read-only token, so this is being handled as robustness hardening.
Expected Behavior
Every annotation emitted by the eval scripts encodes its file property and message the way the GitHub Actions toolkit does, so each annotation is a single, correctly attributed line regardless of the characters in a path or message. The repository already provides this through Write-CIAnnotation in scripts/lib/Modules/CIHelpers.psm1.
Steps to Reproduce
- On Linux, add an eval file whose name contains a comma, for example
evals/sample,line=9.yml, and have it produce a moderation or validation annotation.
- Observe that the annotation's
file property is split at the comma and points at the wrong location.
Related to #3139
Component
Scripts
Bug Description
Several eval scripts under
scripts/evalswrite GitHub Actions workflow commands (::error/::warning) by interpolating file paths and messages directly into the command string, without the percent-encoding GitHub's runner expects. The same applies to two moderation warnings that echo file paths on plain log lines.When a path or message contains characters that are special in workflow commands (
%,:,,, or a line break), the annotation is truncated, attached to the wrong file or line, or split into additional log lines that the runner may interpret. This came up as an out-of-scope observation while reviewing #3139.Affected emitters:
Modules/ModerationRunner.psm1,Invoke-VallyEvals.ps1,Test-EvalSpec.ps1,Test-EvalSpecText.ps1,Test-StimulusPresence.ps1,Test-VallyTestSafety.ps1, andInvoke-ArtifactModeration.ps1.The affected jobs run without secrets and with a read-only token, so this is being handled as robustness hardening.
Expected Behavior
Every annotation emitted by the eval scripts encodes its
fileproperty and message the way the GitHub Actions toolkit does, so each annotation is a single, correctly attributed line regardless of the characters in a path or message. The repository already provides this throughWrite-CIAnnotationinscripts/lib/Modules/CIHelpers.psm1.Steps to Reproduce
evals/sample,line=9.yml, and have it produce a moderation or validation annotation.fileproperty is split at the comma and points at the wrong location.Related to #3139