Skip to content

Track the four new TerminalTestReporter partials in the vendored-files manifest - #55563

Closed
Evangelink wants to merge 1 commit into
dotnet:mainfrom
Evangelink:dev/amauryleve/vendored-reporter-partials
Closed

Track the four new TerminalTestReporter partials in the vendored-files manifest#55563
Evangelink wants to merge 1 commit into
dotnet:mainfrom
Evangelink:dev/amauryleve/vendored-reporter-partials

Conversation

@Evangelink

@Evangelink Evangelink commented Aug 3, 2026

Copy link
Copy Markdown
Member

Adds the four TerminalTestReporter partials that testfx introduced in microsoft/testfx#10387 to the dotnet-test-terminal-reporter entry, so upstream edits to them are visible to the drift detector. Tracked upstream as microsoft/testfx#10390.

Background

testfx split TerminalTestReporter.Summary.cs (715 lines) into focused partial files. The manifest enumerates each upstream partial individually rather than globbing the folder, so the four new paths were silently untracked:

New upstream path Renders
TerminalTestReporter.Coverage.cs code-coverage summary + threshold results
TerminalTestReporter.FlakyTests.cs the "Flaky tests" section
TerminalTestReporter.SlowestTests.cs the "Slowest tests" section
TerminalTestReporter.TestDiscovery.cs discovery-mode output

Nothing changed on the testfx side: its TerminalReporterContract.props includes the reporter via a *.cs glob, so the split needed no manifest edit there. Ours is the manifest that enumerates.

What this PR does

Adds the four sources with "scope": "reporter partial", baselined at testfx main f935d2d3f9ce1ab831a361041b0ceaf92ff73363 with blob SHAs read from the contents API. No existing baseline is touched — this PR is about the missing paths, not about reconciling the outstanding drift.

Why they're appended instead of sorted alphabetically

I originally inserted them in alphabetical order, then caught this with a local check --dry-run: the bot's idempotency marker is <!-- vendored-sync:id={id}:{source-index} -->, i.e. issue identity is the positional index. Inserting .Coverage.cs and .FlakyTests.cs ahead of .Summary.cs shifted it from index 7 to 10, which would have orphaned the open #55472 ((#7)) and #55473 ((#8)) and filed duplicates at the new positions.

So the four are appended, and eng/vendored-files.md now documents that sources is append-only. That felt worth writing down — the trap is invisible unless you happen to run the checker before and after.

One thing worth flagging

While checking what our fork already has, I found that src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs contains no AppendCoverageSummary, AppendFlakyTests, AppendSlowestTests or AppendRetrySummaryLines. That upstream work is not ported.

More importantly, the .Summary.cs drift issue can never surface it. That code was added to Summary.cs upstream after our baseline (dba319b2) and then moved out into the new partials. A unified diff only compares two endpoints, so:

  • at our baseline, Summary.cs had no coverage/flaky/slowest code (not written yet),
  • at current main, Summary.cs has no coverage/flaky/slowest code (moved out),

and the rendered diff therefore shows none of it. Added-then-moved content is invisible to endpoint diffing.

I did not try to encode that backlog in the baseline SHAs — the schema has no way to say "never synced", and faking one would produce a misleading diff. Instead I recorded it in the entry's notes, which the bot already renders into every drift issue for this entry, so it will be in front of whoever picks up #55472. Happy to take a different approach if you'd rather express it structurally.

Validation

validate is clean:

$ python .github/scripts/check_vendored_files.py validate
Manifest OK: 30 entries, 42 sources.

And check --dry-run against live upstream confirms the four new sources resolve, and that every pre-existing index is unchanged:

[drift  ] dotnet-test-terminal-reporter#0
[ok]      dotnet-test-terminal-reporter#1  (TerminalTestReporter.ErroredAssemblies.cs)
[ok]      dotnet-test-terminal-reporter#2  (TerminalTestReporter.Formatting.ControlCharacters.cs)
[ok]      dotnet-test-terminal-reporter#3  (TerminalTestReporter.Formatting.cs)
[ok]      dotnet-test-terminal-reporter#4  (TerminalTestReporter.Handshake.cs)
[ok]      dotnet-test-terminal-reporter#5  (TerminalTestReporter.Lifecycle.cs)
[ok]      dotnet-test-terminal-reporter#6  (TerminalTestReporter.Messaging.cs)
[drift  ] dotnet-test-terminal-reporter#7   <- .Summary.cs, still matches #55472
[drift  ] dotnet-test-terminal-reporter#8   <- .TestCompletion.cs, still matches #55473
[ok]      dotnet-test-terminal-reporter#9  (TerminalTestReporter.Coverage.cs)
[ok]      dotnet-test-terminal-reporter#10 (TerminalTestReporter.FlakyTests.cs)
[ok]      dotnet-test-terminal-reporter#11 (TerminalTestReporter.SlowestTests.cs)
[ok]      dotnet-test-terminal-reporter#12 (TerminalTestReporter.TestDiscovery.cs)

Unrelated, noticed while running the above: check crashes locally on Windows with UnicodeEncodeError: 'charmap' codec can't encode character '\ufeff' when it prints a dry-run body, because testfx's C# files are UTF-8 with BOM. Harmless in CI (UTF-8 default) and I left it alone, but PYTHONIOENCODING=utf-8 is needed to run it locally on Windows.

Copilot AI review requested due to automatic review settings August 3, 2026 08:51
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the eng/vendored-files.json manifest so the vendored-sync drift detector also tracks the four newly-introduced upstream TerminalTestReporter partial files from microsoft/testfx, ensuring upstream edits to those files surface as drift issues in dotnet/sdk.

Changes:

  • Adds four new sources entries under dotnet-test-terminal-reporter for the new upstream partials (Coverage, FlakyTests, SlowestTests, TestDiscovery).
  • Extends the entry’s notes to document known unported upstream functionality that won’t be surfaced by the existing .Summary.cs drift endpoint diff.
Show a summary per file
File Description
eng/vendored-files.json Adds the four new upstream partial paths to the dotnet-test-terminal-reporter vendored entry and updates its notes.

Copilot's findings

  • Files reviewed: 1/1 changed files
  • Comments generated: 1

Comment thread eng/vendored-files.json Outdated
Comment on lines +47 to +70
@@ -52,6 +60,14 @@
"baseline_blob_sha": "9e805c0271f30b6ddf1c729dd286aab5956b52f4",
"scope": "reporter partial"
},
{
"repo": "microsoft/testfx",
"ref": "main",
"path": "src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporter.FlakyTests.cs",
"baseline_ref_sha": "f935d2d3f9ce1ab831a361041b0ceaf92ff73363",
"baseline_blob_sha": "b61b9512cd73c34ecbd86c45abccaeaa5cc9765c",
"scope": "reporter partial"
},
…s manifest

testfx split TerminalTestReporter.Summary.cs into focused partials
(microsoft/testfx#10387), adding four upstream paths that the
dotnet-test-terminal-reporter entry does not list. Until they are tracked,
edits to the coverage, discovery, flaky-test and slowest-test rendering are
invisible to the drift detector.

The new sources are appended rather than sorted into the list: the drift bot
keys issue identity on the positional source index, so inserting would orphan
the open issues for .Summary.cs (dotnet#7) and .TestCompletion.cs (dotnet#8) and file
duplicates at the shifted positions. Document that sources is append-only.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b8858a93-4a8d-4028-b0db-c221ef158206
@Evangelink
Evangelink force-pushed the dev/amauryleve/vendored-reporter-partials branch from 12373e7 to bd1db3a Compare August 3, 2026 08:56
Evangelink added a commit that referenced this pull request Aug 3, 2026
Folded in from the parallel PR #55563: the coverage, flaky-test and slowest-test
sections landed upstream after the .Summary.cs baseline and were then moved into
their own partials, so neither the .Summary.cs drift diff nor the newly tracked
(split-baselined) partials will surface them. Name the backlog in the notes so it
is not lost.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3328d629-8443-45a7-8ee2-97d3ce23dee5
@Evangelink

Copy link
Copy Markdown
Member Author

Closing in favour of #55564, which lands the same four partials plus the fix this PR is missing.

Inserting the sources alphabetically shifts Summary.cs from index 7 to 10 and TestCompletion.cs from 8 to 11. The drift script keys its tracking issues on that index (<!-- vendored-sync:id={id}:{index} -->), so the shift orphans the open issues #55472 (:7) and #55473 (:8) and opens duplicates at the new indices. Worse, index 7 becomes Lifecycle.cs -- the next time it drifts, upsert_issue matches marker :7 and rewrites the Summary drift report in #55472 in place.

#55564 appends the sources instead, so every existing index is preserved.

The one thing this PR had that #55564 did not -- naming the unported coverage/flaky-test/slowest-test backlog in the entry notes -- has been folded into #55564, so nothing is lost here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants