STATUS_UNSUCCESSFUL in RsaCryptRoundtrip_OaepSHA1

[danmoseley]

Not actionable as is.
https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F37960~2Fmerge/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190528.20/workItem/System.Security.Cryptography.Cng.Tests/analysis/xunit/System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span~2FRsaCryptRoundtrip_OaepSHA1

Windows.10.Amd64.ClientRS4.ES.Open-x86-Release
Get Repro environment
Unhandled Exception of Type Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException
Message :
Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : Unknown error (0xc0000001)
Stack Trace :
   at System.Security.Cryptography.RSACng.TryEncryptOrDecrypt(SafeNCryptKeyHandle key, ReadOnlySpan`1 input, Span`1 output, AsymmetricPaddingMode paddingMode, Void* paddingInfo, Boolean encrypt, Int32& bytesWritten) in /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs:line 266
   at System.Security.Cryptography.RSACng.TryEncryptOrDecrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Boolean encrypt, Int32& bytesWritten) in /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs:line 205
   at System.Security.Cryptography.RSACng.TryDecrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Int32& bytesWritten) in /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs:line 38
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.<>c__DisplayClass1_0.<Decrypt>b__0(Byte[] dest) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs:line 16
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.TryWithOutputArray(Func`2 func) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs:line 23
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.Decrypt(RSA rsa, Byte[] data, RSAEncryptionPadding padding) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs:line 16
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip(RSAEncryptionPadding paddingMode, Boolean expectSuccess) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs:line 315
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip_OaepSHA1() in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs:line 285

UPD by @CarnaViire: I went ahead and added Build Information below, from the last occurrence on my PR -- in hopes that known-issues infra would be able to validate and attach the failures not attached yet.

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=565017&view=results
Build error leg or test failing: System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.RsaCryptRoundtrip_OaepSHA512
Pull request:

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "CryptographicException : Unknown error (0x",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=565017
Error message validated: [CryptographicException : Unknown error (0x]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/19/2024 11:07:21 AM UTC

[danmoseley]

Correction, 0xC0000001 (STATUS_UNSUCCESSFUL) not an AV.

[bartonjs]

Oh, the joys of status codes. What's wrong? (UNSUCCESSFUL.) Where? Beats me.

[bartonjs]

Without a stable (or "try this in a loop 30 times") repro or a TTT trace it's hard to bring this up with Windows. Since it's not really actionable right now, marking as Future.

[stephentoub]

Again here:
https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F39369~2Fmerge/test~2Ffunctional~2Fcli~2F~2F/20190710.34/workItem/System.Security.Cryptography.Cng.Tests/analysis/xunit/System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span~2FRsaCryptRoundtrip_OaepSHA512

Windows.10.Amd64.ClientRS4.ES.Open-x64-Debug
Get Repro environment
Unhandled Exception of Type Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException
Message :
Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : Unknown error (0xc0000001)
Stack Trace :
   at System.Security.Cryptography.RSACng.TryEncryptOrDecrypt(SafeNCryptKeyHandle key, ReadOnlySpan`1 input, Span`1 output, AsymmetricPaddingMode paddingMode, Void* paddingInfo, Boolean encrypt, Int32& bytesWritten) in /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs:line 268
   at System.Security.Cryptography.RSACng.TryEncryptOrDecrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Boolean encrypt, Int32& bytesWritten) in /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs:line 205
   at System.Security.Cryptography.RSACng.TryDecrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Int32& bytesWritten) in /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs:line 38
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.<>c__DisplayClass1_0.<Decrypt>b__0(Byte[] dest) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs:line 16
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.TryWithOutputArray(Func`2 func) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs:line 23
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.Decrypt(RSA rsa, Byte[] data, RSAEncryptionPadding padding) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs:line 16
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip(RSAEncryptionPadding paddingMode, Boolean expectSuccess) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs:line 332
   at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip_OaepSHA512() in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs:line 314

[dtivel]

We've hit this issue in several NuGet CI test runs.

[ViktorHofer]

failed again: https://dev.azure.com/dnceng/public/_build/results?buildId=364160&view=ms.vss-test-web.build-test-results-tab&runId=11018888&resultId=100000&paneView=debug

Configuration: netcoreapp-Windows_NT-Debug-x64-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-61052b7-20190723211353

System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.RsaCryptRoundtrip_OaepSHA1 [FAIL]
      Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : Unknown error (0xc0000001)
      Stack Trace:
        /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs(268,0): at System.Security.Cryptography.RSACng.TryEncryptOrDecrypt(SafeNCryptKeyHandle key, ReadOnlySpan`1 input, Span`1 output, AsymmetricPaddingMode paddingMode, Void* paddingInfo, Boolean encrypt, Int32& bytesWritten)
        /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs(205,0): at System.Security.Cryptography.RSACng.TryEncryptOrDecrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Boolean encrypt, Int32& bytesWritten)
        /_/src/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs(38,0): at System.Security.Cryptography.RSACng.TryDecrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Int32& bytesWritten)
        /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs(16,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.<>c__DisplayClass1_0.<Decrypt>b__0(Byte[] dest)
        /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs(23,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.TryWithOutputArray(Func`2 func)
        /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs(16,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Span.Decrypt(RSA rsa, Byte[] data, RSAEncryptionPadding padding)
        /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs(332,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip(RSAEncryptionPadding paddingMode, Boolean expectSuccess)
        /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs(302,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip_OaepSHA1()

@bartonjs can you please take a look? A dump is available.

[bartonjs]

@ViktorHofer I don't see a dump... the test simply failed.

Looking at history on the message over the past 90 days, it looks like one pops up every week or so (sometimes sooner); but there's no real trend (other than it's private key operations (decrypt, sign)).

I think I'm willing to do an experimental retry-once on this error and see if problems go away, now that there's longer to measure it and decide it didn't work or was otherwise a bad idea.

[ViktorHofer]

Oh I must have mixed two builds up, you are right, there's no dump :(

I think I'm willing to do an experimental retry-once on this error and see if problems go away

You mean retrying that operation with the RetryHelper?

[bartonjs]

You mean retrying that operation with the RetryHelper?

No. Making the product code call the Win32 function again if it returns STATUS_UNSUCCESSFUL.

[ViktorHofer]

Trying that out doesn't sound like a bad idea.

[ViktorHofer]

Failed again on UAP: https://dnceng.visualstudio.com/public/_build/results?buildId=366753&view=ms.vss-test-web.build-test-results-tab

[bartonjs]

Sounds like we'll be reverting the retry, then, and that there's very little we can do about it.