[CI] Failures in SamlRealmTests on 6.8

[droberts195]

Build scan:

https://gradle-enterprise.elastic.co/s/3auabo7jtnf6s

Repro line:

./gradlew ':x-pack:plugin:security:unitTest' \
  -Dtests.seed=68F25F624E2A1164 \
  -Dtests.class=org.elasticsearch.xpack.security.authc.saml.SamlRealmTests \
  -Dtests.security.manager=true \
  -Dtests.locale=en-US \
  -Dtests.timezone=UTC \
  -Dcompiler.java=11 \
  -Druntime.java=8

Reproduces locally?:

No (which is quite strange as this seems to make almost every periodic 6.8 build fail across multiple platforms)

Applicable branches:

6.8

Failure history:

https://build-stats.elastic.co/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-30d,mode:quick,to:now))&_a=(columns:!(_source),index:b646ed00-7efc-11e8-bf69-63c8ef516157,interval:auto,query:(language:lucene,query:SamlRealmTests),sort:!(process.time-start,desc))

Failures started on 6th May. Frequency of failures increased on 18th May.

Failure excerpt:

   > Throwable #1: java.security.PrivilegedActionException: net.shibboleth.utilities.java.support.component.ComponentInitializationException: Error refreshing metadata during init
   > 	at __randomizedtesting.SeedInfo.seed([B324C19F0D73FC0C:E9A194C30B6FDA46]:0)
   > 	at java.security.AccessController.doPrivileged(Native Method)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm.initialiseResolver(SamlRealm.java:631)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm.parseHttpMetadata(SamlRealm.java:544)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm.initializeResolver(SamlRealm.java:517)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealmTests.testReadIdpMetadataFromHttps(SamlRealmTests.java:148)
   > 	at java.lang.Thread.run(Thread.java:748)
   > Caused by: net.shibboleth.utilities.java.support.component.ComponentInitializationException: Error refreshing metadata during init
   > 	at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.initMetadataResolver(AbstractReloadingMetadataResolver.java:264)
   > 	at org.opensaml.saml.metadata.resolver.impl.AbstractMetadataResolver.doInitialize(AbstractMetadataResolver.java:287)
   > 	at net.shibboleth.utilities.java.support.component.AbstractInitializableComponent.initialize(AbstractInitializableComponent.java:61)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm.lambda$initialiseResolver$11(SamlRealm.java:632)
   > 	... 42 more
   > Caused by: net.shibboleth.utilities.java.support.resolver.ResolverException: net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://localhost:33951
   > 	at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.refresh(AbstractReloadingMetadataResolver.java:297)
   > 	at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.initMetadataResolver(AbstractReloadingMetadataResolver.java:262)
   > 	... 45 more
   > Caused by: net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from https://localhost:33951
   > 	at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:314)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm$PrivilegedHTTPMetadataResolver.access$001(SamlRealm.java:559)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm$PrivilegedHTTPMetadataResolver.lambda$fetchMetadata$0(SamlRealm.java:569)
   > 	at java.security.AccessController.doPrivileged(Native Method)
   > 	at org.elasticsearch.xpack.security.authc.saml.SamlRealm$PrivilegedHTTPMetadataResolver.fetchMetadata(SamlRealm.java:568)
   > 	at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.refresh(AbstractReloadingMetadataResolver.java:285)
   > 	... 46 more
   > Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
   > 	at sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1470)
   > 	at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1298)
   > 	at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1199)
   > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:401)
   > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:373)
   > 	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
   > 	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
   > 	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
   > 	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
   > 	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
   > 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
   > 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   > 	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   > 	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   > 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   > 	at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:287)
   > 	... 51 more
   > Caused by: java.io.EOFException: SSL peer shut down incorrectly
   > 	at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:480)
   > 	at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:469)
   > 	at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:159)
   > 	at sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
   > 	at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1290)
   > 	... 67 more

This looks like the same error as #30445, although that was fixed years ago.

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[droberts195]

Muted in 6.8 by #73315