[CI] InstallPluginCommandTests fails repeatedly in PR tests and CI

[cbuescher]

I ran into this two times already today in PR testing and also saw failures in CI:

REPRODUCE WITH: ./gradlew :distribution:tools:plugin-cli:test -Dtests.seed=32B7DE2D26697BB -Dtests.class=org.elasticsearch.plugins.InstallPluginCommandTests -Dtests.method="testOfficialPlatformPlugin {p0=com.google.common.jimfs.JimfsFileSystem@5167f57d p1=org.elasticsearch.plugins.InstallPluginCommandTests$1Parameter$$Lambda$17/925973605@64b0598}" -Dtests.security.manager=false -Dtests.locale=is-IS -Dtests.timezone=America/Cancun

I couldn't reproduce it locally so far, also not by removing the "@5167f57d" or the nested Class parts of the test method, however the failures look similar like this:

08:37:25    > Throwable #1: java.io.IOException: Illegal base64 ending sequence:18
08:37:25    > 	at __randomizedtesting.SeedInfo.seed([32B7DE2D26697BB:F90BD3FF60A407F7]:0)
08:37:25    > 	at java.util.Base64$DecInputStream.read(Base64.java:941)
08:37:25    > 	at java.util.Base64$DecInputStream.read(Base64.java:887)
08:37:25    > 	at org.bouncycastle.bcpg.BCPGInputStream.nextPacketTag(Unknown Source)
08:37:25    > 	at org.bouncycastle.openpgp.PGPKeyRing.readOptionalTrustPacket(Unknown Source)
08:37:25    > 	at org.bouncycastle.openpgp.PGPKeyRing.readSignaturesAndTrust(Unknown Source)
08:37:25    > 	at org.bouncycastle.openpgp.PGPKeyRing.readUserIDs(Unknown Source)
08:37:25    > 	at org.bouncycastle.openpgp.PGPPublicKeyRing.<init>(Unknown Source)
08:37:25    > 	at org.bouncycastle.openpgp.PGPObjectFactory.nextObject(Unknown Source)
08:37:25    > 	at org.bouncycastle.openpgp.PGPPublicKeyRingCollection.<init>(Unknown Source)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommand.verifySignature(InstallPluginCommand.java:570)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommandTests$5.verifySignature(InstallPluginCommandTests.java:873)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommand.downloadAndValidate(InstallPluginCommand.java:523)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:257)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:230)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.installPlugin(InstallPluginCommandTests.java:285)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.assertInstallPluginFromUrl(InstallPluginCommandTests.java:921)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.assertInstallPluginFromUrl(InstallPluginCommandTests.java:928)
08:37:25    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.testOfficialPlatformPlugin(InstallPluginCommandTests.java:969)

Failed in PR tests vs. master

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request/10600/consoleFull
and earlier in
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request/10589/consoleFull

Also failed earlier in CI on master at:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+periodic/6384/console
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+g1gc/5910/console

[elasticmachine]

Pinging @elastic/es-core-infra

[cbuescher]

There is another failure on 6.x thats in the same test class, but different method ("testOfficialPlatformPluginStaging" instead of "testOfficialPlatformPlugin"). Looks related though:

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+matrix-java-periodic/ES_BUILD_JAVA=java10,ES_RUNTIME_JAVA=java10,nodes=virtual&&linux/79/consoleFull

REPRODUCE WITH: ./gradlew :distribution:tools:plugin-cli:test \
  -Dtests.seed=D6F67E84B2F3CECF \
  -Dtests.class=org.elasticsearch.plugins.InstallPluginCommandTests \
  -Dtests.method="testOfficialPlatformPluginStaging {p0=com.google.common.jimfs.JimfsFileSystem@7bf9b098 p1=org.elasticsearch.plugins.InstallPluginCommandTests$1Parameter$$Lambda$106/988904418@389adf1d}" \
  -Dtests.security.manager=false \
  -Dtests.locale=zu \
  -Dtests.timezone=Etc/GMT+4

00:32:40    > Throwable #1: java.io.IOException: Illegal base64 ending sequence:18
00:32:40    > 	at __randomizedtesting.SeedInfo.seed([D6F67E84B2F3CECF:AE66D9C9793B5911]:0)
00:32:40    > 	at java.base/java.util.Base64$DecInputStream.read(Base64.java:941)
00:32:40    > 	at java.base/java.util.Base64$DecInputStream.read(Base64.java:887)
00:32:40    > 	at org.bouncycastle.bcpg.BCPGInputStream.nextPacketTag(Unknown Source)
00:32:40    > 	at org.bouncycastle.openpgp.PGPKeyRing.readOptionalTrustPacket(Unknown Source)
00:32:40    > 	at org.bouncycastle.openpgp.PGPKeyRing.readSignaturesAndTrust(Unknown Source)
00:32:40    > 	at org.bouncycastle.openpgp.PGPKeyRing.readUserIDs(Unknown Source)
00:32:40    > 	at org.bouncycastle.openpgp.PGPPublicKeyRing.<init>(Unknown Source)
00:32:40    > 	at org.bouncycastle.openpgp.PGPObjectFactory.nextObject(Unknown Source)
00:32:40    > 	at org.bouncycastle.openpgp.PGPPublicKeyRingCollection.<init>(Unknown Source)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommand.verifySignature(InstallPluginCommand.java:570)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommandTests$5.verifySignature(InstallPluginCommandTests.java:873)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommand.downloadAndValidate(InstallPluginCommand.java:523)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:257)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:230)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.installPlugin(InstallPluginCommandTests.java:285)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.assertInstallPluginFromUrl(InstallPluginCommandTests.java:921)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.assertInstallPluginFromUrl(InstallPluginCommandTests.java:928)
00:32:40    > 	at org.elasticsearch.plugins.InstallPluginCommandTests.testOfficialPlatformPluginStaging(InstallPluginCommandTests.java:985)
00:32:40    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
00:32:40    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
00:32:40    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
00:32:40    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
00:32:40    > 	at java.base/java.lang.Thread.run(Thread.java:844)

[cbuescher]

More https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=sles/2461/console, that time it fails in "testFailedSignatureVerification".

[hendrikmuhs]

I silenced the whole test suite as it seems not to be just 1 test.

[cbuescher]

@hendrikmuhs Since this happens also on 6.x (e.g. https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+multijob-unix-compatibility/os=opensuse/1087/console), we need to silence there as well. Will backport your commit.