HADOOP-17563. Upgrade BouncyCastle to 1.69

[amahussein]

Description of PR

HADOOP-17563 . BouncyCastle to 1.69

• CVEs are reported for releases lower than 1.66

How was this patch tested?

• build locally succeeded mvn clean install -Pdist -Dtar -DskipTests -Dmaven.javadoc
• mvn dependency:tree
• Looked into linked Jiras of HADOOP-15832 and reviewed the dependencies affected by the upgrade
• Some of the the tests below are already timing out on the default branch. However, I verified that they have no class errors as reported in YARN-8919 and YARN-8899

mvn test -Dtest=TestFileArgs,TestMultipleCachefiles,TestStreamingBadRecords,\
TestSymLink,TestMultipleArchiveFiles,TestGridmixSubmission,TestDistCacheEmulation,\
TestLoadJob,TestSleepJob,TestDistCh,TestCleanupAfterKIll

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id
• Updated README.txt

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 52s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 2s		trunk passed
+1 💚	compile	27m 5s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	20m 19s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	25m 41s		trunk passed
+1 💚	javadoc	8m 1s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 59s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	31m 57s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	24m 15s		the patch passed
+1 💚	compile	23m 56s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	23m 56s		the patch passed
+1 💚	compile	19m 47s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 47s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	21m 19s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 47s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	32m 40s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1045m 6s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 33s		The patch does not generate ASF License warnings.
		1317m 16s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
	hadoop.hdfs.server.mover.TestMover

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3405/1/artifact/out/Dockerfile
GITHUB PR	#3405
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml shellcheck shelldocs
uname	Linux 78160c2110eb 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 18d158b
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3405/1/testReport/
Max. process+thread count	2430 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3405/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

[steveloughran]

I'm okay with this, I haven't made any changes to the modules where the tests are failing so don't know how common that is.

We will have to mark this as an incompatible change and explain about the need to upgrade ASM.

[pjfanning]

would it be possible to merge this?

[steveloughran]

ok. should we rebase and retest?

[pjfanning]

@amahussein would it be possible to rebase this? It's not a big deal but bouncycastle 1.70 is now latest version but it may not be worthwhile using that.

[pjfanning]

I created #3980 just in case but can close it in favour of this. At least, we can see if 3980 build succeeds.