Skip to content

Comments

fix: remove global Retry(3) from InvokableTestBase#4925

Merged
thomhurst merged 1 commit intomainfrom
fix/retry-mask
Feb 19, 2026
Merged

fix: remove global Retry(3) from InvokableTestBase#4925
thomhurst merged 1 commit intomainfrom
fix/retry-mask

Conversation

@thomhurst
Copy link
Owner

Summary

  • Removed the global [Retry(3)] attribute from InvokableTestBase, which was silently retrying all ~50 engine test classes and masking intermittent failures
  • Added targeted [Retry(3)] to only the four test classes that are genuinely timing-sensitive:
    • CancellationAfterHooksTests -- validates After hooks run on cancellation, sensitive to process timing
    • ExternalCancellationTests -- sends SIGINT to subprocesses, inherently racy
    • ParallelismValidationEngineTests -- validates concurrency behavior, timing-dependent
    • ConstraintKeySchedulerConcurrencyTests -- high-contention deadlock detection with timeouts

Closes #4878

Test plan

  • dotnet build TUnit.Engine.Tests/TUnit.Engine.Tests.csproj passes with 0 errors
  • CI passes -- the four annotated classes retain retry protection while all other tests now fail fast on first failure

@claude
Copy link
Contributor

claude bot commented Feb 19, 2026

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

This is a well-scoped fix. Removing the blanket [Retry(3)] from InvokableTestBase is the right call — it was silently masking real failures across ~50 test classes. The four classes that received targeted retry attributes (CancellationAfterHooksTests, ExternalCancellationTests, ParallelismValidationEngineTests, ConstraintKeySchedulerConcurrencyTests) are genuinely timing-sensitive and warrant the retry protection. The remaining deterministic tests (subprocess launch → TRX parse → count assertion) are better off failing fast on first failure.

…aky tests

The global [Retry(3)] on InvokableTestBase silently retried all ~50
engine test classes, hiding intermittent failures. Remove it and apply
[Retry(3)] only to the four test classes that are genuinely timing-
sensitive (cancellation hooks, external cancellation, parallelism
validation, constraint-key scheduler concurrency).

Closes #4878
@thomhurst thomhurst merged commit 668cecb into main Feb 19, 2026
13 of 14 checks passed
@thomhurst thomhurst deleted the fix/retry-mask branch February 19, 2026 13:51
This was referenced Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: global Retry(3) on InvokableTestBase masks flaky tests

1 participant