Skip to content

Tests: no test may pass by returning from a catch block - #5254

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/tests-no-silent-pass
Oct 5, 2026
Merged

erikdarlingdata merged 1 commit into
devfrom
fix/tests-no-silent-pass

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

A test of the form try { setup(); } catch { return; } is reported as a pass, not a skip, so where the setup cannot run it asserts nothing and still reads as coverage. A Windows run of a symlink test reported "0 skipped" this way, when creating a symbolic link needs a privilege an ordinary session lacks.

What changed

  • Six sites on dev had a return; inside a catch. Five were Assert.Skip(...) followed by a dead return; (xunit v3's Assert.Skip throws, so these already skipped honestly). The dead return; is gone, so a skip is the only exit. Where the compiler then needed proc assigned, the catch rethrows after the skip call. WebModuleLoadTests is a real test, so it got the same treatment. Three more tests with the same shape turned up when the new guard ran (WebCollectionHealthRunAtColumnTests, WebDatabaseSizesNoteColumnTests, AzureSqlDatabaseHardwareTests) and got the same fix. StoreThatRefusesTheLogin_ExitsWithTheStoreCode returns from a catch inside a listener helper loop, not from the test, so it is left alone.
  • The certificate symlink test now proves the link resolves (File.ResolveLinkTarget(..., returnFinalTarget: true)) before asserting the refusal.
  • New ExportViewerConfigAsync_JunctionAtTheDestinationOrTheCertificatePath_RefusesWithoutWritingThrough: a directory junction (mklink /J) needs no privilege, so this case cannot skip for want of a right. It asserts the junction exists and resolves, then that the export refuses and writes nothing through it, for a junction as the destination and a junction at server.crt.
  • New TestsNoSilentPassGuardTests: scans both test projects and fails if any [Fact]/[Theory] has a bare return; inside a catch. The message names Assert.Skip / Assert.SkipUnless. It has a self-test of the scanner and no exemption list.

Verified

  • Build Release, 0 warnings, 0 errors.
  • Guard family (90 classes) plus the touched classes: Total 1224, Failed 5, Skipped 58. The failures are the expected platform ones on this Mac: four ResolveViewerExportDirectory_* path tests (Windows paths) and LivePostgresCollectionHygieneTests. They fail on dev too.
  • The new guard and its self-test pass. hygiene-scan: 0 offenders.
  • Executed on the Mac: the guard, the Node-based web tests (Node is installed), the toggle exit-code test. Skipped on the Mac (DPAPI requires Windows): the certificate symlink test and the new junction test. CI on Windows decides those.
  • Reading the product: a junction destination is refused at DarlingCliCommands.cs:1224, and a junction at server.crt has the ReparsePoint attribute, so it takes the symbolic-link refusal at :1672. Not run.

CHANGELOG entry

SECTION: None
ENTRY: None: test-only change, no user-visible effect

Replace the dead return after Assert.Skip with a rethrow so the skip is the
only exit, add a junction case for the certificate path that needs no
privilege, and add a guard that fails on a bare return in a catch of a test.
@erikdarlingdata erikdarlingdata added the merge-train In a merge train: its CI ran on the train PR label Oct 5, 2026
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 5, 2026 14:52
@erikdarlingdata
erikdarlingdata merged commit 388f686 into dev Oct 5, 2026
13 of 16 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/tests-no-silent-pass branch October 5, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-train In a merge train: its CI ran on the train PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant