Skip to content

NpgsqlRootCertificateValidationTests.ChainShape_VerifyFullWithPrintedRoot fails locally on Windows (environmental) #3557

Closed

Description

@erikdarlingdata

Found while running the full Darling.Tests suite for #3532 (schedule clamp work): ChainShape_VerifyFullWithPrintedRoot_CompletesTheHandshake_OnEveryPlatform fails on the local dev box, in isolation and in full runs.

Pre-existing, not a regression: the Sep 12 build of Darling.Tests in the main repo checkout (predating the #3532 branch entirely) fails the same test identically on this machine. CI is green on dev, so this is machine-environmental, not shape-intrinsic.

The test drives a real Npgsql VerifyFull connect against an in-test TLS listener; the sibling legacy-shape test's dynamic skip reports handshake completed = True on the same run, so the listener plumbing works — it's specifically the chain-shape serving path (the SChannel ephemeral-key PFX round-trip in HandshakeCompletesAsync) that reports completed=false locally. Worth a look next time TLS/#2117 code is touched; every other Darling.Tests unit passes (9702/9703).

The rest of the suite is unaffected and the failure reproduces with Darling.Tests.exe -method "*ChainShape_VerifyFullWithPrintedRoot*".

Activity

  1. erikdarlingdata commented on Sep 18, 2026

    @erikdarlingdata
    OwnerAuthor

    Root cause found and fixed in #3572. Short version: this was never Npgsql rejecting the chain - the harness's fake server died before TLS even started, and the harness swallowed the exception, leaving only completed = false to misread.

    Mechanism: Windows caches every root a TLS client processes into that user's CurrentUser\CA store. Every Darling store install/rotation mints a fresh root under the same subject, and this box's dogfooding cycles deposited ~55 copies of CN=PerformanceMonitor Darling store root (localhost) there between Aug 10 and Sep 8. At that pile size, CryptoAPI's subject-matched issuer walk errors out (CryptographicException: An unknown chain building error occurred) - killing SslStreamCertificateContext.Create in the test harness, deterministically on the first colliding build in a fresh process. CI is green because fresh runners have no pile. The test was also arming its own failure: every green run deposited one more same-subject root (proven thumbprint-for-thumbprint during diagnosis).

    Fix (#3572): a random mint tag in the generated root's CN makes every rotation a distinct CA, so the pile can never form - on any machine, including field Windows viewers (a fresh-process probe showed the default-trust build a viewer's SslStream runs is exposed the same way). SKI/AKI was implemented and tested first and does not prevent the engine error; subject uniqueness does. Validated 6/6 fresh-process runs plus a full green suite with the pile deliberately left in place.

    Machine state left for Erik to decide (nothing deleted): the ~55 stale cached roots are still in CurrentUser\CA. They are no longer needed for test green, but the deployed DARLING01 store keeps serving its existing same-subject chain until its cert rotates (delete cert+key, restart), so local default-trust chain builds can still hit the engine error until cleanup or rotation. Cleanup one-liner if wanted:

    Get-ChildItem Cert:\CurrentUser\CA | Where-Object Subject -like '*Darling store root*' | Remove-Item
  2. added 2 commits that reference this issue on Sep 18, 2026
  3. erikdarlingdata commented on Sep 18, 2026

    @erikdarlingdata
    OwnerAuthor

    Fixed on dev in PR #3572 (unique per-generation root CN). Closing per the dev-merge convention. Machine-state follow-up (the ~55 stale cached roots in CurrentUser\CA and the DARLING01 store-cert rotation) is tracked in the root-cause comment above and is operator-side, not a code gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions