Consider running System.Security.Cryptography tests on Windows 10 Threshold

[vcsjones]

It appears that the JIT will run some of its tests in a pipeline with Windows 10 / Windows Server 2016 SP0 (Threshold). However, this pipeline will not be triggered for Libraries-only changes.

System.Security.Cryptography is a little sensitive to OS versions, particularly around PKCS12/PFX. There are known differences between Windows 8, Threshold, and Redstone 2+. This has led to a few cases where a test was green and merged because it ran on "new" Windows, only to later find out that the test fails on Windows 10 Threshold when a JIT pipeline runs.

Examples:

• System.Security.Cryptography.X509Certificates.Tests.PfxTests.ReadMLKem512PrivateKey_NotSupported failing with CryptographicException #115156
• System.Security.Cryptography.X509Certificates.Tests.CollectionTests "Assert.Equal() Failure: Values differ" #112738
• ReadECDsaPrivateKey_BrainpoolP160r1_Pfx test failure on windows #108815
• ReadSlhDsa_Pfx_Ietf_NotSupported: CryptographicException : The specified network password is not correct. #115270

Ideally, Windows 10 SP0 / Threshold would be used for library runs that touch cryptography, or a reasonable as possible. A set of path filters might look like

- src/libraries/System.Security.Cryptography*/
- src/libraries/Common/src/System/Security/Cryptography/

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[vcsjones]

/cc @dotnet/area-system-security

[ericstj]

Here are the set of different libraries test runs for windows -

runtime/eng/pipelines/libraries/helix-queues-setup.yml

Lines 103 to 125 in 43955cb

	# windows x64
	- ${{ if eq(parameters.platform, 'windows_x64') }}:
	# netcoreapp
	- ${{ if notIn(parameters.jobParameters.framework, 'net481') }}:
	# libraries on mono outerloop
	- ${{ if and(eq(parameters.jobParameters.testScope, 'outerloop'), eq(parameters.jobParameters.runtimeFlavor, 'mono')) }}:
	- Windows.Amd64.Server2022.Open
	- Windows.Server2025.Amd64.Open
	# libraries on coreclr (outerloop and innerloop), or libraries on mono innerloop
	- ${{ if or(ne(parameters.jobParameters.testScope, 'outerloop'), ne(parameters.jobParameters.runtimeFlavor, 'mono')) }}:
	- ${{ if or(eq(parameters.jobParameters.isExtraPlatformsBuild, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
	- Windows.Amd64.Server2022.Open
	- Windows.Server2025.Amd64.Open
	- ${{ if ne(parameters.jobParameters.testScope, 'outerloop') }}:
	- (Windows.10.Amd64.ServerRS5.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:windowsservercore-ltsc2019-helix-amd64
	- ${{ if or(ne(parameters.jobParameters.isExtraPlatformsBuild, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
	- Windows.Amd64.Server2022.Open
	- Windows.Server2025.Amd64.Open
	- Windows.11.Amd64.Client.Open
	- ${{ if eq(parameters.jobParameters.testScope, 'outerloop') }}:
	- (Windows.10.Amd64.ServerRS5.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:windowsservercore-ltsc2019-helix-amd64
	- ${{ if ne(parameters.jobParameters.runtimeFlavor, 'mono') }}:
	- (Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64

Do you see something here that would cover this? Maybe outerloop or extra-platforms?

[dotnet-policy-service]

This issue has been marked needs-author-action and may be missing some important information.