Repository navigation
[Test Optimization] Add NUnit 5 support - #9407
Open
tonyredondo wants to merge 12 commits into
Open
tonyredondo wants to merge 12 commits into
tonyredondo wants to merge 12 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![datadog-datadog-prod-us1-2[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Dependency-skipped fixtures report every discovered descendant, including tests excluded by execution filters, producing inflated test totals and skip counts for filtered NUnit 5 runs.
🤖 Bits Code Review · Commit d61edb7
andrewlock
reviewed
Oct 14, 2026
tonyredondo
marked this pull request as draft
October 14, 2026 16:55
BenchmarksBenchmark execution time: 2026-10-15 12:10:05 Comparing candidate commit d125d3d in PR branch Found 0 performance improvements and 12 performance regressions! Performance is the same for 60 metrics, 0 unstable metrics, 74 known flaky benchmarks, 52 flaky benchmarks without significant changes.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (9407) and master. ✅ No regressions detected |
…ement - Hide only failures of NUnit 5 prerequisites for quarantine and Attempt to Fix; hidden failures still block the dependents of their prerequisite suites. - Inspect the dependency graph only for tests ITR would skip. - Clear stale native Retry errors when the final result is skipped or inconclusive. - Report leaf dependency skips without scanning their siblings.
…heap - Only marked prerequisite suites reach the ConditionalWeakTable lock on completion. - Filter dependency skips by their result state before creating result proxies.
- Add per-scenario runner arguments to TestScenario. - Drop the no-op NUnit.PreFilter=false and run fixture dependency scenarios without NUnit workers.
- Replace the CanHideResult predicate, which recorded hidden failures, with TryHideResult, which also hides the result. - Skip error cleanup for spans without errors and clarify the dependency-skip hook comment.
tonyredondo
marked this pull request as ready for review
October 15, 2026 12:43
Contributor
There was a problem hiding this comment.
More details
Static review found no concrete regression in dependency scheduling, skipped-test reporting, or final-result error cleanup. Runtime validation was blocked by the unavailable .NET 11 preview SDK.
🤖 Bits Code Review · Commit d125d3d
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d125d3d294
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Add NUnit 5 support to Test Optimization, including dependency skips, ITR and test management decisions that preserve NUnit dependency semantics, and passing native Repeat/Retry results.
Reason for change
NUnit 5 keeps the existing execution signatures, but dependency skips bypass child execution/completion hooks, and NUnit decides which dependents run from prerequisite results that ITR and test management can rewrite. Its AssemblyVersion is fixed at 5.0.0.0 (NUnit 4.6.1 also ships 4.6.0.0), so it does not identify the package version.
Implementation details
Test coverage
Other details