Skip to content

Conversation

steveloughran
Copy link
Contributor

How was this patch tested?

Going to see if it works on a mac...

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

Contributed by Steve Loughran
@steveloughran
Copy link
Contributor Author

the test which is parameterized on ssl (and storediag when a store is forced to OpenSSL)

[ERROR] org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek.testReadFullyZeroBytebufferPastEOF -- Time elapsed: 0.570 s <<< ERROR!
java.io.IOException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: openssl.TLS, provider: openssl, class: org.wildfly.openssl.OpenSSLContextSPI$OpenSSLTLSContextSpi)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.<init>(DelegatingSSLSocketFactory.java:137)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.initializeDefaultFactory(DelegatingSSLSocketFactory.java:106)
        at org.apache.hadoop.fs.s3a.impl.NetworkBinding.bindSSLChannelMode(NetworkBinding.java:81)
        at org.apache.hadoop.fs.s3a.impl.AWSClientConfig.createHttpClientBuilder(AWSClientConfig.java:168)
        at org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createS3Client(DefaultS3ClientFactory.java:141)
        at org.apache.hadoop.fs.s3a.impl.ClientManagerImpl.lambda$createS3Client$0(ClientManagerImpl.java:137)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
        at org.apache.hadoop.util.functional.LazyAtomicReference.eval(LazyAtomicReference.java:94)
        at org.apache.hadoop.util.functional.LazyAutoCloseableReference.eval(LazyAutoCloseableReference.java:54)
        at org.apache.hadoop.fs.s3a.impl.ClientManagerImpl.getOrCreateS3Client(ClientManagerImpl.java:178)
        at org.apache.hadoop.fs.s3a.impl.S3AStoreImpl.getOrCreateS3Client(S3AStoreImpl.java:340)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:814)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3615)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:554)
        at org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
        at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:181)
        at org.apache.hadoop.fs.contract.AbstractContractSeekTest.setup(AbstractContractSeekTest.java:59)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at java.util.ArrayList.forEach(ArrayList.java:1259)
        at java.util.Optional.ifPresent(Optional.java:159)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
        at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at java.util.ArrayList.forEach(ArrayList.java:1259)
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: openssl.TLS, provider: openssl, class: org.wildfly.openssl.OpenSSLContextSPI$OpenSSLTLSContextSpi)
        at java.security.Provider$Service.newInstance(Provider.java:1686)
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
        at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.bindToOpenSSLProvider(DelegatingSSLSocketFactory.java:207)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.initializeSSLContext(DelegatingSSLSocketFactory.java:170)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.<init>(DelegatingSSLSocketFactory.java:135)
        ... 58 more
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at org.wildfly.openssl.SSL.init(SSL.java:87)
        at org.wildfly.openssl.OpenSSLContextSPI.<init>(OpenSSLContextSPI.java:137)
        at org.wildfly.openssl.OpenSSLContextSPI$OpenSSLTLSContextSpi.<init>(OpenSSLContextSPI.java:453)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.security.Provider$Service.newInstance(Provider.java:1664)
        ... 64 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.wildfly.openssl.SSL.init(SSL.java:82)
        ... 68 more
Caused by: java.lang.UnsatisfiedLinkError: no wfssl in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1875)
        at java.lang.Runtime.loadLibrary0(Runtime.java:844)
        at java.lang.System.loadLibrary(System.java:1136)
        at org.wildfly.openssl.SSL$LibraryLoader.load(SSL.java:288)
        ... 70 more

Move to the 2.25 and my mbp still fails

Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: openssl.TLS, provider: openssl, class: org.wildfly.openssl.OpenSSLContextSPI$OpenSSLTLSContextSpi)
        at java.base/java.security.Provider$Service.newInstance(Provider.java:1879)
        at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
        at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
        at java.base/javax.net.ssl.SSLContext.getInstance(SSLContext.java:185)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.bindToOpenSSLProvider(DelegatingSSLSocketFactory.java:207)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.initializeSSLContext(DelegatingSSLSocketFactory.java:170)
        at org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.<init>(DelegatingSSLSocketFactory.java:135)
        ... 31 more
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at org.wildfly.openssl.SSL.init(SSL.java:87)
        at org.wildfly.openssl.OpenSSLContextSPI.<init>(OpenSSLContextSPI.java:137)
        at org.wildfly.openssl.OpenSSLContextSPI$OpenSSLTLSContextSpi.<init>(OpenSSLContextSPI.java:453)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at java.base/java.security.Provider$Service.newInstanceOf(Provider.java:1890)
        at java.base/java.security.Provider$Service.newInstanceUtil(Provider.java:1897)
        at java.base/java.security.Provider$Service.newInstance(Provider.java:1872)
        ... 37 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.wildfly.openssl.SSL.init(SSL.java:82)
        ... 47 more
Caused by: java.lang.UnsatisfiedLinkError: no wfssl in java.library.path: /Users/stevel/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
        at java.base/java.lang.System.loadLibrary(System.java:1989)
        at org.wildfly.openssl.SSL$LibraryLoader.load(SSL.java:288)
        ... 52 more
2025-10-08 13:39:23,489 [main] INFO  util.ExitUtil (ExitUtil.java:terminate(248)) - Exiting with status -1: java.io.IOException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: openssl.TLS, provider: openssl, class: org.wildfly.openssl.OpenSSLContextSPI$OpenSSLTLSContextSpi)

@steveloughran
Copy link
Contributor Author

s3a tests all good, s3 london -Dparallel-tests -DtestsThreadCount=8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant