Skip to content

fix(scripts): eval annotations interpolate paths into workflow commands without encoding #3153

Description

Component

Scripts

Bug Description

Several eval scripts under scripts/evals write GitHub Actions workflow commands (::error / ::warning) by interpolating file paths and messages directly into the command string, without the percent-encoding GitHub's runner expects. The same applies to two moderation warnings that echo file paths on plain log lines.

When a path or message contains characters that are special in workflow commands (%, :, ,, or a line break), the annotation is truncated, attached to the wrong file or line, or split into additional log lines that the runner may interpret. This came up as an out-of-scope observation while reviewing #3139.

Affected emitters: Modules/ModerationRunner.psm1, Invoke-VallyEvals.ps1, Test-EvalSpec.ps1, Test-EvalSpecText.ps1, Test-StimulusPresence.ps1, Test-VallyTestSafety.ps1, and Invoke-ArtifactModeration.ps1.

The affected jobs run without secrets and with a read-only token, so this is being handled as robustness hardening.

Expected Behavior

Every annotation emitted by the eval scripts encodes its file property and message the way the GitHub Actions toolkit does, so each annotation is a single, correctly attributed line regardless of the characters in a path or message. The repository already provides this through Write-CIAnnotation in scripts/lib/Modules/CIHelpers.psm1.

Steps to Reproduce

  1. On Linux, add an eval file whose name contains a comma, for example evals/sample,line=9.yml, and have it produce a moderation or validation annotation.
  2. Observe that the annotation's file property is split at the comma and points at the wrong location.

Related to #3139

Activity

  1. added a commit that references this issue on Oct 9, 2026
    d763f71
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingneeds-triageRequires triage and prioritization

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions