fix(evals): keep the leading dot in moderation record ids by MohammedAlkindi · Pull Request #3139 · microsoft/hve-core

Mohammed Alkindi (MohammedAlkindi) · 2026-10-05T22:31:20Z

Pull Request

Description

Moderation ids lost the leading dot of .github because TrimStart strips every leading dot and slash. Now only one leading ./ or .\ is dropped.

Related Issue(s)

Fixes #3132

Type of Change

Bug fix (non-breaking change fixing an issue)
Script/automation (.ps1, .sh, .py)

Sample Prompts (for AI Artifact Contributions)

N/A

Testing

31 Pester tests pass; the dot-directory and relative-root cases fail on main. validate:local passes except lint:md-links.

Checklist

Required Checks

Documentation is updated (if applicable)
Files follow existing naming conventions
Changes are backwards compatible (if applicable)
Tests added for new functionality (if applicable)

AI Artifact Contributions

N/A

Required Local Checks

Local validation aggregate: npm run validate:local
Documentation validation (if docs changed): npm run validate:docs (N/A)
Spell checking: npm run spell-check
Link validation: npm run lint:md-links (N/A)

Security Considerations

This PR does not contain any sensitive or NDA information
Any new dependencies have been reviewed for security issues
Security-related scripts follow the principle of least privilege

Additional Notes

None.

Bill Berry (WilliamBerryiii)

Thank you for this fix, Mohammed Alkindi (@MohammedAlkindi), and for the clear root-cause write-up in #3132. The anchored regex is the right approach. We verified it on Windows (pwsh 7.6) and Linux (pwsh 7.5): all 29 tests pass on both, the dot-directory and relative-root cases fail against main, and PSScriptAnalyzer is clean.

One thing before merge:

Please run npm run validate:local and npm run spell-check and check those boxes. The docs validation, link validation, and "Documentation is updated" items can be marked N/A, since no Markdown changed.

The inline comments are optional test and consistency suggestions. Please comment if you have questions about any of them, and we can discuss.

Bill Berry (WilliamBerryiii) · 2026-10-06T21:43:24Z

Thanks again for this fix, Mohammed Alkindi (@MohammedAlkindi). While reviewing it, we noticed that the eval scripts write annotation file paths into workflow commands without encoding them. That is now tracked in #3153 and addressed in #3154. Since #3154 also touches ModerationRunner.psm1 a few lines from your change, whichever PR merges second may need a quick rebase. No action is needed from you.

Mohammed Alkindi (MohammedAlkindi) · 2026-10-07T05:41:19Z

Done in 901711e: NestedId hoisted with your separator comment, count guards before indexing, the outside-root and ..foo cases as separate tests, and root.md asserted in the relative-root test. 31 tests pass. I left the forward-slash normalization out, since #3154 changes the lines next to it. npm run validate:local passes every step except lint:md-links, which fails on two dead external links (an owasp.org 404 and a medium.com 403) in files this PR does not touch.

Codecov Comments Bot (codecov-commenter) · 2026-10-07T18:55:17Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.01%. Comparing base (d763f71) to head (cef82a1).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3139      +/-   ##
==========================================
+ Coverage   83.98%   84.01%   +0.03%     
==========================================
  Files          95      192      +97     
  Lines       13715    38813   +25098     
  Branches        0      201     +201     
==========================================
+ Hits        11518    32609   +21091     
- Misses       2197     6133    +3936     
- Partials        0       71      +71

Flag	Coverage Δ
pester	`83.86% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 104 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

# Pull Request ## Description Routes every GitHub Actions annotation emitted by the eval scripts through the shared `Write-CIAnnotation` helper, so file paths and messages are percent-encoded per the runner's workflow-command rules instead of being interpolated raw. * **Annotation encoding**: all 43 raw `::error` / `::warning` emitters across seven files in `scripts/evals` now call `Write-CIAnnotation` with the same level, file, line, and message text. `%`, CR, LF, `:`, and `,` are encoded, and `\` in file paths becomes `/`. * **Path-echo warnings**: the moderation `File not found` and `Artifact file not found` warnings are now emitted as `::warning::` annotations instead of plain warning lines that echo untrusted paths. * **Module dependency**: `ModerationRunner.psm1` imports CIHelpers (without `-Force`, matching `SecurityHelpers.psm1`), and each converted script imports it before its dot-source guard. * **Tests**: * exact-output tests for hostile record IDs and file values * line-mapping call-site tests for `Test-EvalSpecText` and `Test-VallyTestSafety` (new test file) * an encoded-warning test for `Invoke-ArtifactModeration` * a static guard that fails if a raw workflow-command emitter is reintroduced under `scripts/evals` * **Test determinism**: the one Vally test that asserts raw annotation text now pins `GITHUB_ACTIONS=true` for its child process, so it passes locally as well as in CI. The threshold-override fixture now copies CIHelpers alongside `ModerationRunner.psm1`. Outside CI, `Write-CIAnnotation` reports annotations through `Write-Warning`, so local runs show `WARNING: ERROR [file] message` instead of a raw `::error` line. Exit codes and JSON outputs are unchanged. ## Related Issue(s) Fixes microsoft#3153 Related to microsoft#3139 ## Type of Change Select all that apply: **Code & Documentation:** * [x] Bug fix (non-breaking change fixing an issue) * [ ] New feature (non-breaking change adding functionality) * [ ] Breaking change (fix or feature causing existing functionality to change) * [ ] Documentation update **Infrastructure & Configuration:** * [ ] GitHub Actions workflow * [ ] Linting configuration (markdown, PowerShell, etc.) * [ ] Security configuration * [ ] DevContainer configuration * [ ] Dependency update **AI Artifacts:** * [ ] Reviewed contribution with `hve-builder` and addressed all actionable findings * [ ] Copilot instructions (`.github/instructions/*.instructions.md`) * [ ] Copilot prompt (`.github/prompts/*.prompt.md`) * [ ] Copilot agent (`.github/agents/*.agent.md`) * [ ] Copilot skill (`.github/skills/*/SKILL.md`) * [ ] Copilot hook (`.github/hooks/*/*.json`) * [ ] Eval spec added/updated for changed AI artifacts (`evals/`) > Note for AI Artifact Contributors: > > * Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review `.github/agents/` before creating new ones. > * Skills: Must include both bash and PowerShell scripts. See [Skills](../docs/contributing/skills.md). > * Model Versions: Contributions **MUST** target models listed in the model catalog (`scripts/linting/model-catalog.json`) whose provider appears in `providerAllowlist` and whose status is `ga` or `preview`. Run `npm run lint:models` to validate references. > * See [Agents Not Accepted](../docs/contributing/custom-agents.md#agents-not-accepted) and [Model Version Requirements](../docs/contributing/ai-artifacts-common.md#model-version-requirements). **Other:** * [x] Script/automation (`.ps1`, `.sh`, `.py`) * [ ] Other (please describe): ## Testing All runs used the committed branch. * Full eval suite, Unit plus Integration (`npm run test:ps -- -TestPath "scripts/tests/evals/" -ExcludeTag Slow`): * Windows: 1396 passed, 0 failed, with `GITHUB_ACTIONS` both unset and set to `true`. * Linux (WSL, pwsh 7.5.4): 1395 passed, 1 failed, in both modes. The one failure (`diagnostics.checkout` is null) is a WSL-only artifact: WSL git can't resolve this git worktree's Windows-path `gitdir`. That code path is unchanged and passes on Windows and in normal checkouts. * `npm run test:ps -- -TestPath "scripts/tests/lib/CIHelpers.Tests.ps1"`: 85 passed. * `npm run lint:ps`: passed. * `npm run spell-check`: passed (0 issues). * `git diff --check`: clean. * Regression proofs: * Reintroducing an inline raw emitter fails the new guard test. * Reverting the `Artifact file not found` conversion fails its new test. * Injecting a spurious warning fails both no-warning moderation tests. ## Checklist ### Required Checks * [ ] Documentation is updated (if applicable) * [x] Files follow existing naming conventions * [x] Changes are backwards compatible (if applicable) * [x] Tests added for new functionality (if applicable) ### AI Artifact Contributions  * [ ] Used `hve-builder` review mode to review contribution * [ ] Addressed all actionable findings from the `hve-builder` review * [ ] Verified contribution follows common standards and type-specific requirements ### Required Local Checks The following local-safe validation commands must pass before merging: * [ ] Local validation aggregate: `npm run validate:local` * [ ] Documentation validation (if docs changed): `npm run validate:docs` * [x] Spell checking: `npm run spell-check` * [ ] Link validation: `npm run lint:md-links` ## Security Considerations  * [ ] This PR does not contain any sensitive or NDA information * [ ] Any new dependencies have been reviewed for security issues * [ ] Security-related scripts follow the principle of least privilege No dependencies changed. The change hardens annotation output; it adds no permissions, network access, or credential handling. ## Additional Notes * Documentation: no docs describe the annotation format, and `docs/contributing/evals-ci.md` ("emits a single `::error file=...::` annotation per missing artifact") remains accurate. * `npm run validate:local` stops at `lint:md-links`, which reports 5 unreachable external URLs in unchanged Markdown (`SUPPORT.md`, `TRANSPARENCY-NOTE.md`, and three skill or instruction references). I ran the remaining 22 steps individually. 21 passed. `lint:py` couldn't run because the per-skill uv environments aren't synced in this worktree, and no Python changed. * Emitters outside `scripts/evals` (for example `scripts/security/Install-PSModules.ps1`) are out of scope and can follow in a separate sweep. * `ModerationRunner.psm1` is also touched by microsoft#3139 (a different line, three lines away), so whichever merges second may need a trivial rebase. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix(evals): keep the leading dot in moderation record ids

97c4b61

Mohammed Alkindi (MohammedAlkindi) requested a review from a team as a code owner October 5, 2026 22:31

Bill Berry (WilliamBerryiii) reviewed Oct 6, 2026

View reviewed changes

This was referenced Oct 6, 2026

fix(scripts): eval annotations interpolate paths into workflow commands without encoding #3153

Closed

fix(scripts): encode eval workflow-command annotations #3154

Merged

test(evals): cover outside-root and double-dot moderation ids

901711e

Merge branch 'main' into fix/moderation-record-id-dot-dirs

1e598cd

Chris Montazer (rezatnoMsirhC) enabled auto-merge October 7, 2026 18:46

Chris Montazer (rezatnoMsirhC) approved these changes Oct 7, 2026

View reviewed changes

Merge branch 'main' into fix/moderation-record-id-dot-dirs

cef82a1

Bill Berry (WilliamBerryiii) approved these changes Oct 9, 2026

View reviewed changes

Chris Montazer (rezatnoMsirhC) added this pull request to the merge queue Oct 9, 2026

Merged via the queue into microsoft:main with commit a664a17 Oct 9, 2026
147 checks passed

Navigation Menu

Repository navigation

fix(evals): keep the leading dot in moderation record ids - #3139

fix(evals): keep the leading dot in moderation record ids#3139
Chris Montazer (rezatnoMsirhC) merged 4 commits into
microsoft:mainfrom
MohammedAlkindi:fix/moderation-record-id-dot-dirs

Mohammed Alkindi (MohammedAlkindi) commented Oct 5, 2026 •

edited

Loading

Uh oh!

Bill Berry (WilliamBerryiii) left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Bill Berry (WilliamBerryiii) commented Oct 6, 2026

Uh oh!

Mohammed Alkindi (MohammedAlkindi) commented Oct 7, 2026

Uh oh!

Codecov Comments Bot (codecov-commenter) commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Mohammed Alkindi (MohammedAlkindi) commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bill Berry (WilliamBerryiii) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Bill Berry (WilliamBerryiii) commented Oct 6, 2026

Uh oh!

Mohammed Alkindi (MohammedAlkindi) commented Oct 7, 2026

Uh oh!

Codecov Comments Bot (codecov-commenter) commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Mohammed Alkindi (MohammedAlkindi) commented Oct 5, 2026 •

edited

Loading

Codecov Comments Bot (codecov-commenter) commented Oct 7, 2026 •

edited

Loading