Error exporting https certificate on macOS Sonoma 14.4

[holm-dk]

Something changed in macOS 14.4 (compared to 14.3) that caused PEM cert export to start raising Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK).

The original repro uses dotnet dev-certs to generate and export a self-signed cert aspnetcore uses during local development, but I've included a self-contained repro below.

[Original description below]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Generating https development certificates using this commands throws an exception.

dotnet dev-certs https -v -ep ./test.pem --format Pem --no-password

Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK)

Full output:

[1] Listing certificates from CurrentUser\My
[2] Found certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[3] Checking certificates validity
[4] Valid certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[5] Invalid certificates: no certificates
[6] Finished listing certificates.
[1] Listing certificates from CurrentUser\My
[2] Found certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[3] Checking certificates validity
[4] Valid certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[5] Invalid certificates: no certificates
[6] Finished listing certificates.
[1] Listing certificates from LocalMachine\My
[2] Found certificates: no certificates
[3] Checking certificates validity
[4] Valid certificates: no certificates
[5] Invalid certificates: no certificates
[6] Finished listing certificates.
[8] Filtered certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[9] Excluded certificates: no certificates
[14] Valid certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[15] Selected certificate: 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[23] Saving certificate '9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true' to ./test.pem with private key.
[24] Exporting certificate with private key but no password.
[25] Creating directory ..
[26] An error has occurred while exporting the certificate: Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK)
   at Interop.AppleCrypto.TrySecKeyCopyExternalRepresentation(SafeSecKeyRefHandle key, Byte[]& externalRepresentation)
   at System.Security.Cryptography.RSAImplementation.RSASecurityTransforms.ExportParameters(Boolean includePrivateParameters)
   at System.Security.Cryptography.RSA.WritePkcs1PrivateKey()
   at System.Security.Cryptography.RSA.TryExportRSAPrivateKey(Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.RSA.WritePkcs8PrivateKey()
   at System.Security.Cryptography.RSA.TryExportEncryptedPkcs8PrivateKey(ReadOnlySpan`1 password, PbeParameters pbeParameters, Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.AsymmetricAlgorithm.ExportArray[T](ReadOnlySpan`1 password, PbeParameters pbeParameters, TryExportPbe`1 exporter)
   at Microsoft.AspNetCore.Certificates.Generation.CertificateManager.ExportCertificate(X509Certificate2 certificate, String path, Boolean includePrivateKey, String password, CertificateKeyExportFormat format).
[26] An error has occurred while exporting the certificate: Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK)
   at Interop.AppleCrypto.TrySecKeyCopyExternalRepresentation(SafeSecKeyRefHandle key, Byte[]& externalRepresentation)
   at System.Security.Cryptography.RSAImplementation.RSASecurityTransforms.ExportParameters(Boolean includePrivateParameters)
   at System.Security.Cryptography.RSA.WritePkcs1PrivateKey()
   at System.Security.Cryptography.RSA.TryExportRSAPrivateKey(Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.RSA.WritePkcs8PrivateKey()
   at System.Security.Cryptography.RSA.TryExportEncryptedPkcs8PrivateKey(ReadOnlySpan`1 password, PbeParameters pbeParameters, Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.AsymmetricAlgorithm.ExportArray[T](ReadOnlySpan`1 password, PbeParameters pbeParameters, TryExportPbe`1 exporter)
   at Microsoft.AspNetCore.Certificates.Generation.CertificateManager.ExportCertificate(X509Certificate2 certificate, String path, Boolean includePrivateKey, String password, CertificateKeyExportFormat format)
   at Microsoft.AspNetCore.Certificates.Generation.CertificateManager.EnsureAspNetCoreHttpsDevelopmentCertificate(DateTimeOffset notBefore, DateTimeOffset notAfter, String path, Boolean trust, Boolean includePrivateKey, String password, CertificateKeyExportFormat keyExportFormat, Boolean isInteractive).
There was an error exporting HTTPS developer certificate to a file.

.NET Version

8.0.201

Anything else?

> dotnet --info
.NET SDK:
 Version:           8.0.201
 Commit:            4c2d78f037
 Workload version:  8.0.200-manifests.3097af8b

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  14.4
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/8.0.201/

.NET workloads installed:
There are no installed workloads to display.

Host:
  Version:      8.0.2
  Architecture: arm64
  Commit:       1381d5ebd2

.NET SDKs installed:
  8.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  x64   [/usr/local/share/dotnet/x64]

Environment variables:
  Not set

global.json file:
  Not found

[cpboyd]

Also facing the same issue. Removing --no-password allows the command to succeed, but this option is used in several SPA projects generated by Visual Studio.

Edit: Ah, the issue is with exporting the private .key file. Without --no-password, it doesn't generate the .key file.
Specifying --password test instead results in the same error.
Even trying to run the commands as sudo still results in the same error.

[cpboyd]

Ok. I've found a workaround.
First, find the trusted cert file:

ls ~/.aspnet/dev-certs/https

Next, use openssl to extract the info (replace <cert-ID> with the unique ID):

openssl pkcs12 -in ~/.aspnet/dev-certs/https/aspnetcore-localhost-<cert-ID>.pfx -out cert.pem -legacy -nodes

When prompted for a password, just press Return/Enter.

Then you need to further split the PRIVATE KEY block into its own file <filename>.key:

-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----

And the CERTIFICATE as well into a matching <filename>.cert:

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

It's not very straight-forward, but it seems like dotnet dev-certs has been deprecated and hasn't seen updates in quite some time: https://www.nuget.org/packages/dotnet-dev-certs

[holm-dk]

Ok. I've found a workaround.

You can also generate the self-signed SSL certificate for localhost and key using openssl:

openssl req -x509 -sha256 -nodes -newkey rsa:2048 -days 365 -subj '/CN=localhost' -keyout localhost.key -out localhost.pem

It's not very straight-forward, but it seems like dotnet dev-certs has been deprecated and hasn't seen updates in quite some time: https://www.nuget.org/packages/dotnet-dev-certs

Only that NuGet package is deprecated, dotnet dev-certs is not (it's no longer a "tool").

[cpboyd]

@thj-dk I guess the main thing that the dotnet cert is supposed to add over that is that it's trusted by the system. (Which, yes, you could manually trust the cert generated via openssl as well.)

[amcasey]

Yep, the in-box version of dotnet dev-certs is still very much supported, though the tool itself is fairly simple and doesn't change very often.

The call in question appears to be https://github.com/dotnet/aspnetcore/blob/702ba39ace967037a3f726765732e3e63f80907e/src/Shared/CertificateGeneration/CertificateManager.cs#L506

Unfortunately, Apple appears to use that error code for a number of different conditions, making it hard to debug by inspection. We may need to pull in https://github.com/dotnet/runtime for their understanding of the PAL.

[amcasey]

I tried pulling out the important calls from dev-certs. Does this throw the same exception?

using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;

using var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
var certs = store.Certificates.OfType<X509Certificate2>().ToList();
var cert = certs.Single(c => c.FriendlyName == "ASP.NET Core HTTPS development certificate");
var key = cert.GetRSAPrivateKey()!;
var eyBytes = key.ExportEncryptedPkcs8PrivateKey(string.Empty, new PbeParameters(PbeEncryptionAlgorithm.Aes256Cbc, HashAlgorithmName.SHA256, 1));

I'd ask for a copy of the cert, but exporting seems to be the crux of the issue. 😆

[amcasey]

Huh, a colleague happened to be on 14.3 and it didn't repro there, but it did start failing in this way after upgrading to 14.4. Didn't see that coming.

[amcasey]

Self-contained repro

using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Text;

var cert = Repro.CreateCertificate();
var key = cert.GetRSAPrivateKey()!;
var eyBytes = key.ExportEncryptedPkcs8PrivateKey(string.Empty, new PbeParameters(PbeEncryptionAlgorithm.Aes256Cbc, HashAlgorithmName.SHA256, 1));

public class Repro
{
    internal const int CurrentAspNetCoreCertificateVersion = 2;
    internal const string AspNetHttpsOid = "1.3.6.1.4.1.311.84.1.1";
    internal const string AspNetHttpsOidFriendlyName = "ASP.NET Core HTTPS development certificate";

    private const string ServerAuthenticationEnhancedKeyUsageOid = "1.3.6.1.5.5.7.3.1";
    private const string ServerAuthenticationEnhancedKeyUsageOidFriendlyName = "Server Authentication";

    private const string LocalhostHttpsDnsName = "localhost";
    private const string LocalhostHttpsDistinguishedName = "CN=" + LocalhostHttpsDnsName;

    public const int RSAMinimumKeySizeInBits = 2048;

    public static X509Certificate2 CreateCertificate()
    {
        // shims
        var notBefore = DateTime.Now;
        var notAfter = DateTime.Now + TimeSpan.FromDays(365);
        var Subject = LocalhostHttpsDistinguishedName;
        var AspNetHttpsCertificateVersion = CurrentAspNetCoreCertificateVersion;


        var subject = new X500DistinguishedName(Subject);
        var extensions = new List<X509Extension>();
        var sanBuilder = new SubjectAlternativeNameBuilder();
        sanBuilder.AddDnsName(LocalhostHttpsDnsName);

        var keyUsage = new X509KeyUsageExtension(X509KeyUsageFlags.KeyEncipherment | X509KeyUsageFlags.DigitalSignature, critical: true);
        var enhancedKeyUsage = new X509EnhancedKeyUsageExtension(
            new OidCollection() {
                new Oid(
                    ServerAuthenticationEnhancedKeyUsageOid,
                    ServerAuthenticationEnhancedKeyUsageOidFriendlyName)
            },
            critical: true);

        var basicConstraints = new X509BasicConstraintsExtension(
            certificateAuthority: false,
            hasPathLengthConstraint: false,
            pathLengthConstraint: 0,
            critical: true);

        byte[] bytePayload;

        if (AspNetHttpsCertificateVersion != 0)
        {
            bytePayload = new byte[1];
            bytePayload[0] = (byte)AspNetHttpsCertificateVersion;
        }
        else
        {
            bytePayload = Encoding.ASCII.GetBytes(AspNetHttpsOidFriendlyName);
        }

        var aspNetHttpsExtension = new X509Extension(
            new AsnEncodedData(
                new Oid(AspNetHttpsOid, AspNetHttpsOidFriendlyName),
                bytePayload),
            critical: false);

        extensions.Add(basicConstraints);
        extensions.Add(keyUsage);
        extensions.Add(enhancedKeyUsage);
        extensions.Add(sanBuilder.Build(critical: true));
        extensions.Add(aspNetHttpsExtension);

        var certificate = CreateSelfSignedCertificate(subject, extensions, notBefore, notAfter);
        return certificate;
    }

    internal static X509Certificate2 CreateSelfSignedCertificate(
            X500DistinguishedName subject,
            IEnumerable<X509Extension> extensions,
            DateTimeOffset notBefore,
            DateTimeOffset notAfter)
    {
        using var key = CreateKeyMaterial(RSAMinimumKeySizeInBits);

        var request = new CertificateRequest(subject, key, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
        foreach (var extension in extensions)
        {
            request.CertificateExtensions.Add(extension);
        }

        var result = request.CreateSelfSigned(notBefore, notAfter);
        return result;

        RSA CreateKeyMaterial(int minimumKeySize)
        {
            var rsa = RSA.Create(minimumKeySize);
            if (rsa.KeySize < minimumKeySize)
            {
                throw new InvalidOperationException($"Failed to create a key with a size of {minimumKeySize} bits");
            }

            return rsa;
        }
    }
}

[vcsjones]

Thanks for the self-contained repro @amcasey. I'll take a look.