The CI guard tests hand-parse workflow YAML, and now do it twice #255

Problem / motivation

FormCraft.UnitTests/Ci/ now holds two guard suites that read the same files the same way, with their
own copies of the code that does it.

TrustedPublishingWorkflowTests.cs (#173, #198, #226) and TestReportingTests.cs (added in #252) each
declare their own:

• RepoRoot / LocateRepoRoot() — walk up from AppContext.BaseDirectory until FormCraft.sln
• WorkflowsDirectory, ReadWorkflow(), ReadBuildScript()
• WithoutComments(text, marker) — drop whole-line comments so a "not referenced" assertion fires on
  wiring rather than on prose
• an independent enumeration of .yml/.yaml under .github/workflows

These are byte-identical including their explanatory comments — both carry the same
dir.Parent is DirectoryInfo?, so the variable has to be too note and the same
a trailing-comment strip would also mangle the "https://" in a URL justification. And
TestReportingTests.StepNamed() is a one-line variant of TrustedPublishingWorkflowTests.StepWithId():
both find a step by scanning from its header line to the next line whose trimmed text starts with
- , differing only in whether the header is matched on - name: or on id:.

This is not a tidiness complaint. That step-boundary scan is an approximation of YAML, shared by
both files and owned by neither. A fix to it now has to be made twice or the copies drift — and this
codebase has already paid that price: #205 records a copied test comment that propagated a factual
error (AddCommonFieldAttributes described as "shared by the text and numeric paths") and hid the date
path from coverage until review caught it. Duplicated scaffolding duplicates its errors too.

The gap the duplication is holding open

Both suites' assertions are file-scoped, not job-scoped, and closing that is the same piece of work.

WorkflowsThatRunTests() matches on file content, and StepNamed() scans the whole file. So a
workflow that ran ./build.cmd Test in job A and uploaded the test-results artifact from job B would
satisfy every assertion in TestReportingTests while uploading nothing at all — job B gets a fresh
workspace, so test-results/ never exists there. Combined with if-no-files-found: ignore (which
#252 pinned on all three uploads), that failure is completely silent.

This is not hypothetical: release-please.yml is already a two-job workflow (release-please and
nupkg). It is correct today only because the build and the upload happen to sit in the same job.
Structurally it is the same "declared but inert" shape #231 was filed about, moved from the artifact
path to the job boundary.

Catching it needs the workflow split into jobs before the step scan runs — which is exactly the kind of
thing that should exist once, in a shared reader, rather than being added to one copy and forgotten in
the other. The limitation is currently recorded only as a <remarks> block on TestReportingTests.

This was raised in #252's code review and deliberately deferred there, to keep a refactor of a passing
guard suite out of a build/CI fix.

Proposed solution

Extract a single FormCraft.UnitTests/Ci/WorkflowSource helper owning repo-root location, workflow
enumeration and reading (cached — the files cannot change during a run), comment stripping, and step
extraction. Move both existing classes onto it. Then give it job awareness and scope
TestReportingTests' step assertions to the job that actually runs the build.

Job splitting is done by indentation rather than by adding a YAML parser: jobs: is a top-level key
and each job is its own two-space-indented key, which is a small, testable rule. release-please.yml
is the fixture that proves it — a real two-job workflow already in the repo.

Alternatives considered

• Leave it. The duplication is only 2× today and both suites pass. But Ci/ has grown a file per
  CI investigation (Pin the version-tag publish rule: IsOnVersionTag is unanchored by design but untested #227 added VersionTagRuleTests.cs, ci: emit test reports natively under Microsoft.Testing.Platform (#231) #252 added this one) and the third copy is the
  natural next step. Leaving it also leaves the job-scoping hole open, which is the part that can go
  silently wrong.
• Add a real YAML parser (YamlDotNet) and drop the hand-rolled scanning entirely. Genuinely more
  correct, and worth doing if the assertions get much more structural. Rejected for now: packages are
  centrally pinned in Directory.Packages.props, and adding a dependency to assert on five files the
  repo controls is a poor trade — especially as the tests deliberately assert on text in places
  (WithoutComments exists so a documentation edit cannot redden the suite, a distinction a parsed
  model discards).
• Merge the two suites into one class. Removes the duplication trivially, but loses the
  one-suite-per-concern layout that makes it clear which tests guard which issue — the same reasoning
  Extract a shared collection-item test fixture instead of copying models per suite #205 gives for keeping its suites separate.

Area

Test infrastructure — FormCraft.UnitTests/Ci/. No library code, no workflow or Build.cs behaviour
change.

Related: #231, #252 (introduced the second copy), #198 and #226 (own the other suite), #205 (the
same duplicated-scaffolding pattern in the MudBlazor test project), #227.

internal static class WorkflowSource
    string  RepoRoot                                  // located once, cached
    string  WorkflowsDirectory
    IReadOnlyDictionary<string,string> All            // filename -> raw text, read once
    string  Read(string fileName)
    string  ReadBuildScript()
    string  WithoutComments(string text, string marker)
    IReadOnlyList<string> Matching(string pattern)    // workflow files whose stripped text matches
    IReadOnlyDictionary<string,string> JobsOf(string fileName)   // job name -> that job's text
    string  StepNamed(string scope, string stepName)  // scope = a whole file's text, or one job's
    string  StepWithId(string scope, string stepId)

flowchart TD
    A[workflow text] --> B{line == 'jobs:' at col 0}
    B -->|no| A
    B -->|yes| C[scan forward]
    C --> D{line matches '^  [A-Za-z0-9_-]+:'}
    D -->|yes| E[close previous job, open new one]
    D -->|no| F{indent == 0 and non-blank}
    F -->|yes| G[end of jobs block]
    F -->|no| C

🛠️ Implementation plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: one shared workflow reader for the Ci/ guard suites, then job-scoped step assertions.

Architecture: test-infrastructure only, inside FormCraft.UnitTests/Ci/. These suites assert on
repository files (.github/workflows/*.yml, build/Build.cs) rather than on library behaviour.

Tech stack: net10.0, xUnit v3 on Microsoft.Testing.Platform, Shouldly, Arrange-Act-Assert.

Global Constraints

• Base branch is dev, not main. The PR targets dev.
• Commit identity: Philippe Matray <phmatray@gmail.com>.
• TreatWarningsAsErrors=true is deliberate — never relax it to make a build pass. Watch nullability:
  DirectoryInfo? dir in LocateRepoRoot is annotated the way it is for this reason.
• dotnet test --filter is inert under MTP (warning MTP0001) — always run the full suite
  (dotnet test -c Release) and never report a filtered result.
• No new NuGet package. Versions are centrally pinned in Directory.Packages.props; adding a YAML
  parser is explicitly out of scope here (see Alternatives considered).
• No change to any workflow, to build/Build.cs, or to library code. If an assertion would require
  editing a workflow to pass, stop — that is a finding, not a refactor.
• CHANGELOG.md is owned by release-please — never hand-edit it.

Task 1: Extract WorkflowSource and move TestReportingTests onto it

Files: create FormCraft.UnitTests/Ci/WorkflowSource.cs; modify
FormCraft.UnitTests/Ci/TestReportingTests.cs.

Interfaces: internal static class WorkflowSource exposing RepoRoot, WorkflowsDirectory, All,
Read(string), ReadBuildScript(), WithoutComments(string,string), Matching(string),
StepNamed(string scope, string stepName), StepWithId(string scope, string stepId).

• Step 1: Write FormCraft.UnitTests/Ci/WorkflowSourceTests.cs asserting the primitives against
  the real repo: All contains every .yml under .github/workflows, WithoutComments drops a
  leading-# line but keeps a URL containing //, and StepNamed returns only the named step's
  own lines (assert it excludes the following step's - name:). Run dotnet test -c Release and
  confirm RED (type does not exist).
• Step 2: Implement WorkflowSource, reading all workflow files once into a static dictionary.
  Move the bodies of LocateRepoRoot, WithoutComments and the step scan across unchanged.
• Step 3: Run dotnet test -c Release; confirm the new suite is GREEN.
• Step 4: Rewrite TestReportingTests to delegate to WorkflowSource, deleting its private
  copies. Do not change any assertion.
• Step 5: Run dotnet test -c Release; confirm GREEN with the same test count as before
  plus the new WorkflowSourceTests.
• Step 6: Commit: test(ci): extract a shared workflow reader for the CI guard suites

Task 2: Move TrustedPublishingWorkflowTests onto the shared reader

Files: modify FormCraft.UnitTests/Ci/TrustedPublishingWorkflowTests.cs.

Interfaces: no new surface — StepWithId and StepCondition become thin wrappers over
WorkflowSource.

• Step 1: Confirm the suite is green before touching it, so any later red is attributable:
  run dotnet test -c Release and record the pass count.
• Step 2: Delete this class's copies of RepoRoot/LocateRepoRoot(), WorkflowsDirectory,
  ReadWorkflow(), ReadBuildScript() and WithoutComments(), delegating to WorkflowSource.
  Keep StepCondition — it is specific to this suite's if:-vs-with: distinction (Decide whether a missing NUGET_USER should silently skip the release publish #226).
• Step 3: Run dotnet test -c Release; confirm GREEN with an unchanged pass count.
• Step 4: Verify no copy survives: grep -rn "LocateRepoRoot\|WithoutComments" FormCraft.UnitTests/Ci/
  should show definitions only in WorkflowSource.cs.
• Step 5: Commit: test(ci): move the trusted-publishing guards onto the shared workflow reader

Task 3: Split workflows into jobs and scope the test-results assertions to one

Files: modify FormCraft.UnitTests/Ci/WorkflowSource.cs,
FormCraft.UnitTests/Ci/WorkflowSourceTests.cs, FormCraft.UnitTests/Ci/TestReportingTests.cs.

Interfaces: add IReadOnlyDictionary<string,string> JobsOf(string fileName).

• Step 1: Write the failing test for JobsOf: release-please.yml splits into exactly
  release-please and nupkg, the nupkg job's text contains ./build.cmd Pack, and the
  release-please job's text does not. Run dotnet test -c Release → RED.
• Step 2: Implement JobsOf by indentation, applying WithoutComments first so a commented-out
  invocation cannot mark a job as test-running.
• Step 3: Run dotnet test -c Release → GREEN.
• Step 4: Rework TestReportingTests' five workflow assertions to iterate (workflow, job) pairs
  whose job text invokes ./build.[cmd|sh|ps1] (Test|Pack|PublishIfNeeded|Publish|Continuous|Release),
  asserting the Publish: test-results step exists in that same job and satisfies the existing
  four checks. Keep the vacuity guard on the pair set.
• Step 5: Run dotnet test -c Release → GREEN (the repo satisfies this today: the build and
  the upload both live in release-please.yml's nupkg job).
• Step 6: Prove the new scoping is not vacuous — temporarily move ci.yml's
  Publish: test-results step into a second job, confirm the suite turns RED, then revert.
• Step 7: Delete the <remarks> paragraph on TestReportingTests recording the file-scoped
  limitation, which no longer applies.
• Step 8: Commit: test(ci): scope the test-results assertions to the job that runs the build