Java 13 (I know...) Compatibility - XDH Key Generation (Works on Java 12)

[brcolow]

I am not sure if this has anything to do with Bouncycastle. Java 13 has JEP 353 which rewrites (parts) of the legacy socket API. I am using Bouncycastle 1.62.

When I try to connect to Amazon AWS DynamoDB on Java 13, I get the following (partial) stack trace:

Caused by: java.lang.RuntimeException: Could not generate XDH keypair
at java.base/sun.security.ssl.XDHKeyExchange$XDHEPossession.<init>(XDHKeyExchange.java:110)
at java.base/sun.security.ssl.NamedGroup$XDHFunctions.createPossession(NamedGroup.java:750)
at java.base/sun.security.ssl.NamedGroup.createPossession(NamedGroup.java:390)
at java.base/sun.security.ssl.SSLKeyExchange$T13KeyAgreement.createPossession(SSLKeyExchange.java:568)
at java.base/sun.security.ssl.SSLKeyExchange.createPossessions(SSLKeyExchange.java:84)
at java.base/sun.security.ssl.KeyShareExtension$CHKeyShareProducer.produce(KeyShareExtension.java:255)
at java.base/sun.security.ssl.SSLExtension.produce(SSLExtension.java:571)
at java.base/sun.security.ssl.SSLExtensions.produce(SSLExtensions.java:250)
at java.base/sun.security.ssl.ClientHello$ClientHelloKickstartProducer.produce(ClientHello.java:649)
at java.base/sun.security.ssl.SSLHandshake.kickstart(SSLHandshake.java:519)
at java.base/sun.security.ssl.ClientHandshakeContext.kickstart(ClientHandshakeContext.java:107)
at java.base/sun.security.ssl.TransportContext.kickstart(TransportContext.java:231)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:430)
... 61 more
Caused by: java.security.InvalidAlgorithmParameterException: invalid parameterSpec: java.security.spec.NamedParameterSpec@15e0fe05
at org.bouncycastle.jcajce.provider.asymmetric.edec.KeyPairGeneratorSpi.initialize(Unknown Source)
at java.base/java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:699)
at java.base/sun.security.ssl.XDHKeyExchange$XDHEPossession.<init>(XDHKeyExchange.java:105)
... 73 more

Downgrading to the latest nightly build of the Java 12 repository makes the error go away.

The actual chain of events here is that the Amazon AWS SDK is calling Apache's HTTP client which is trying to make an SSL connection. I am not 100% sure how Bouncycastle comes into it unless the only XDH provider accessible is Bouncycastle? I tried to put the provider at position 2, but it made no difference.

I believe the full stack trace where a custom AWS Lambda runtime calls a Lambda request handler, calls DynamoDB, calls Apache, calls, etc. etc. is essentially obfuscation. I think this would show up trying to make an SSL connection to Amazon's DynamoDB server that happens to use XDH key exchange. I see some changes to the sun.security.ssl.SSLSocketImpl classes (as well as other classes in the stack trace) new to Java 13 (most likely related to the aforementioned JEP).

This could very well be a bug in the new Java socket implementation and if so I will certainly open this upstream.

Thanks for your time.

[brcolow]

Click to expand (Full Stack Trace)

Caused by: javax.net.ssl.SSLException: Could not generate XDH keypair
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
at java.base/sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1652)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:443)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
at software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory.connectSocket(SdkTlsSocketFactory.java:113)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:80)
at com.sun.proxy.$Proxy5.connect(Unknown Source)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:240)
at software.amazon.awssdk.http.apache.ApacheHttpClient.access$500(ApacheHttpClient.java:106)
at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:221)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:66)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:51)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:35)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:64)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.doExecute(RetryableStage.java:113)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:86)
... 27 more
Caused by: java.lang.RuntimeException: Could not generate XDH keypair
at java.base/sun.security.ssl.XDHKeyExchange$XDHEPossession.<init>(XDHKeyExchange.java:110)
at java.base/sun.security.ssl.NamedGroup$XDHFunctions.createPossession(NamedGroup.java:750)
at java.base/sun.security.ssl.NamedGroup.createPossession(NamedGroup.java:390)
at java.base/sun.security.ssl.SSLKeyExchange$T13KeyAgreement.createPossession(SSLKeyExchange.java:568)
at java.base/sun.security.ssl.SSLKeyExchange.createPossessions(SSLKeyExchange.java:84)
at java.base/sun.security.ssl.KeyShareExtension$CHKeyShareProducer.produce(KeyShareExtension.java:255)
at java.base/sun.security.ssl.SSLExtension.produce(SSLExtension.java:571)
at java.base/sun.security.ssl.SSLExtensions.produce(SSLExtensions.java:250)
at java.base/sun.security.ssl.ClientHello$ClientHelloKickstartProducer.produce(ClientHello.java:649)
at java.base/sun.security.ssl.SSLHandshake.kickstart(SSLHandshake.java:519)
at java.base/sun.security.ssl.ClientHandshakeContext.kickstart(ClientHandshakeContext.java:107)
at java.base/sun.security.ssl.TransportContext.kickstart(TransportContext.java:231)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:430)
... 61 more
Caused by: java.security.InvalidAlgorithmParameterException: invalid parameterSpec: java.security.spec.NamedParameterSpec@7318daf8
at org.bouncycastle.jcajce.provider.asymmetric.edec.KeyPairGeneratorSpi.initialize(Unknown Source)
at java.base/java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:699)
at java.base/sun.security.ssl.XDHKeyExchange$XDHEPossession.<init>(XDHKeyExchange.java:105)
... 73 more

[peterdettman]

So it turns out that JDK 13 builds have added support for X25519 and X448 to the SunJSSE provider: https://bugs.openjdk.java.net/browse/JDK-8171279 . The SunJSSE provider relies on other provider(s) to supply KeyPairGenerator, KeyFactory and KeyAgreement engines.

BC provider has implementations for all these, but unfortunately SunJSSE is using java.security.spec.NamedParameterSpec (a type introduced in JDK 11) as the AlgorithmParameterSpec when initialising the X25519/X448 KeyPairGenerator, and BC doesn't recognize/support it (yet), and so the error.

Lowering the priority of the BC provider should avoid this, but it would need to be moved below the SunEC provider in the list.

Another option is to set the system property jdk.tls.namedGroups so that x25519 and x448 are not enabled. e.g.:
jdk.tls.namedGroups="secp256r1, secp384r1, ffdhe2048, ffdhe3072"
(this covers the most common named groups besides X25519/X448, but you may need to add more).

Thanks for the early report of a looming issue; we'll add NamedParameterSpec support for our next release.

[brcolow]

Thanks for the information and fast response. So essentially we need to add an override method to
https://github.com/bcgit/bc-java/blob/25b8c29939f4c80ae42f274a74bf6f299bc9c084/prov/src/main/java/org/bouncycastle/jcajce/provider/asymmetric/edec/KeyPairGeneratorSpi.java ?

Namely:

   @Override
  public void initialize(AlgorithmParameterSpec params,
                SecureRandom random) {
        NamedParameterSpec nps = (NamedParameterSpec) params;
        // Etc...
   }

If this is the right approach I could try and submit a PR.

[peterdettman]

Well we already have that method, but it needs to also recognise NamedParameterSpec. That's relatively trivial; the complication is that NamedParameterSpec was added in JDK 11 but our jars have to run on earlier JDKs still. It's probably best to just leave it with us for now.

[brcolow]

Gotcha. Thanks again.

[brcolow]

Even with:

    static {
        if (Security.getProvider(BouncyCastleProvider.PROVIDER_NAME) == null) {
            int requestedPosition = 2;
            int actualPosition = Security.insertProviderAt(new BouncyCastleProvider(), 9999);
            System.out.println("Requested to add BouncyCastleProvider at position: " + requestedPosition +
                    " and was actually added at position: " + actualPosition);
        }
    }

The Bouncycastle provider is still being called and the same error is resulting.

[brcolow]

The System.setProperty("jdk.tls.namedGroups", "secp256r1, secp384r1, ffdhe2048, ffdhe3072"); work-around does indeed work.

[peterdettman]

OK, glad that's working. Moving the provider to after SunEC provider should also work; the code snippet shown above does not convince me that you tested this properly.

[brcolow]

Is there a way to query what "slot" the SunEC provider is in to make sure that BouncyCastle is after that number?

[peterdettman]

Well you can see the list in your java.security config file, and even install BC that way. If it has to be programmatic, then look at the other methods available on java.security.Security class, in particular the getProviders method will return all installed providers "in their preference order".

Okay, I think this is now fixed, I've added handling of any algorithm parameter spec that has a getName() method. This will appear in 1.64.

[spadou]

Hi, using 1.64 I have the following exception:

Caused by: java.lang.ClassCastException: class org.bouncycastle.jcajce.provider.asymmetric.edec.BCXDHPublicKey cannot be cast to class java.security.interfaces.XECPublicKey (org.bouncycastle.jcajce.provider.asymmetric.edec.BCXDHPublicKey is in unnamed module of loader 'app'; java.security.interfaces.XECPublicKey is in module java.base of loader 'bootstrap')
	at java.base/sun.security.ssl.XDHKeyExchange$XDHEPossession.<init>(XDHKeyExchange.java:108)
	at java.base/sun.security.ssl.NamedGroup$XDHFunctions.createPossession(NamedGroup.java:750)
	at java.base/sun.security.ssl.NamedGroup.createPossession(NamedGroup.java:390)
	at java.base/sun.security.ssl.SSLKeyExchange$T13KeyAgreement.createPossession(SSLKeyExchange.java:568)
	at java.base/sun.security.ssl.SSLKeyExchange.createPossessions(SSLKeyExchange.java:84)
	at java.base/sun.security.ssl.KeyShareExtension$CHKeyShareProducer.produce(KeyShareExtension.java:255)
	at java.base/sun.security.ssl.SSLExtension.produce(SSLExtension.java:571)
	at java.base/sun.security.ssl.SSLExtensions.produce(SSLExtensions.java:250)
	at java.base/sun.security.ssl.ClientHello$ClientHelloKickstartProducer.produce(ClientHello.java:649)
	at java.base/sun.security.ssl.SSLHandshake.kickstart(SSLHandshake.java:519)
	at java.base/sun.security.ssl.ClientHandshakeContext.kickstart(ClientHandshakeContext.java:107)
	at java.base/sun.security.ssl.TransportContext.kickstart(TransportContext.java:231)
	at java.base/sun.security.ssl.SSLEngineImpl.beginHandshake(SSLEngineImpl.java:107)
	... 1 more

Look like the key exchange implementation expect a XECPublicKey interface for the public key.

[brcolow]

@SamuelPadou I don't think that is related to this issue.

[spadou]

It's not exactly the same error but it seem related. It is also linked to the added support for X25519 and X448 by JDK-8171279 in the JDK13 build.

The fix above in 1.64 correct the exception in the KeyPairGenerator initialisation, but this other exception occurs 2 lines after when the XDHEPossession try to cast the public key to XECPublicKey. This interface was also introduced in JDK 11 and it look like BC doesn't support it either.

I'll do some more tests and open another issue if this isn't related to this one.