HDFS-17542. EC: Optimize the EC block reconstruction.

[zhengchenyu]

Description of PR

https://issues.apache.org/jira/browse/HDFS-17542

The current reconstruction process of EC blocks is based on the original contiguous blocks. It is mainly implemented through the work constructed by computeReconstructionWorkForBlocks. It can be roughly divided into three processes:

• scheduleReconstruction
• chooseTargets
• validateReconstructionWork

For ordinary contiguous blocks:

• (1) scheduleReconstruction
  Select srcNodes as the source of the copy block according to the status of each replica of the block.
• (2) chooseTargets
  Select the target of the copy.
• (3) validateReconstructionWork
  Add the copy command to srcNode, srcNode receives the command through heartbeat, and executes the block copy from src to target.

For EC blocks:
(1) and (2) seems nearly same. However, whether to perform simple block copy or block reconstruction for EC blocks is determined in (3). And when some storage is busy, may result no work, it will lead to the problem described in HDFS-17516. Even if no block copying or block reconstruction is generated, pendingReconstruction and neededReconstruction will still be updated until the block times out, which wastes the scheduling opportunity.

Because the decision of whether to perform block copy or block reconstruction is made in (3), unnecessary liveBusyBlockIndices, and excludeReconstructedIndices are introduced. We know many bugs are related here. These should be avoided.

How was this patch tested?

unit test and test in cluster

For code changes:

• Move the work of deciding whether to copy or reconstruct blocks from validateReconstructionWork to scheduleReconstruction.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	11m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	45m 7s		trunk passed
+1 💚	compile	1m 20s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	1m 10s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	1m 7s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 49s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 29s		trunk passed
+1 💚	shadedclient	37m 20s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 9s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	1m 12s		the patch passed
+1 💚	compile	1m 4s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	1m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 0s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 116 unchanged - 1 fixed = 121 total (was 117)
+1 💚	mvnsite	1m 17s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 37s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
-1 ❌	spotbugs	3m 25s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	37m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	231m 50s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 45s		The patch does not generate ASF License warnings.
		384m 47s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	org.apache.hadoop.hdfs.server.blockmanagement.NumberReplicasStriped doesn't override org.apache.hadoop.hdfs.util.EnumCounters.equals(Object) At NumberReplicasStriped.java:At NumberReplicasStriped.java:[line 1]
Failed junit tests	hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
	hadoop.hdfs.server.blockmanagement.TestDatanodeManager

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/1/artifact/out/Dockerfile
GITHUB PR	#6915
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 116b1dfefa0d 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 535d712
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/1/testReport/
Max. process+thread count	4086 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 51s		trunk passed
+1 💚	compile	1m 22s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	1m 20s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	1m 16s		trunk passed
+1 💚	mvnsite	1m 26s		trunk passed
+1 💚	javadoc	1m 8s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 43s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 23s		trunk passed
+1 💚	shadedclient	36m 41s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	1m 18s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	1m 18s		the patch passed
+1 💚	compile	1m 10s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	1m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 5s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 116 unchanged - 1 fixed = 121 total (was 117)
+1 💚	mvnsite	1m 18s		the patch passed
+1 💚	javadoc	0m 56s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 36s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
-1 ❌	spotbugs	3m 28s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	37m 12s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	232m 52s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		374m 37s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	org.apache.hadoop.hdfs.server.blockmanagement.NumberReplicasStriped doesn't override org.apache.hadoop.hdfs.util.EnumCounters.equals(Object) At NumberReplicasStriped.java:At NumberReplicasStriped.java:[line 1]
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestDatanodeManager

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/2/artifact/out/Dockerfile
GITHUB PR	#6915
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 37abfc76f55a 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / f1ac383
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/2/testReport/
Max. process+thread count	4217 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 29s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	45m 20s		trunk passed
+1 💚	compile	1m 22s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	1m 17s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	1m 14s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	1m 8s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 48s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 18s		trunk passed
+1 💚	shadedclient	35m 53s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
-1 ❌	mvninstall	0m 24s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	compile	0m 23s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt	hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.
-1 ❌	javac	0m 23s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt	hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.
-1 ❌	compile	0m 25s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt	hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.
-1 ❌	javac	0m 25s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt	hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 58s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 116 unchanged - 1 fixed = 118 total (was 117)
-1 ❌	mvnsite	0m 26s	/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	javadoc	0m 24s	/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt	hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.
-1 ❌	javadoc	0m 24s	/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt	hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.
-1 ❌	spotbugs	0m 24s	/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	shadedclient	11m 43s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 26s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		105m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/3/artifact/out/Dockerfile
GITHUB PR	#6915
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux cb6430637470 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / f6c3215
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/3/testReport/
Max. process+thread count	646 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
-1 ❌	mvninstall	45m 1s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	1m 21s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	1m 11s		trunk passed
+1 💚	mvnsite	1m 24s		trunk passed
+1 💚	javadoc	1m 8s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 45s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 19s		trunk passed
+1 💚	shadedclient	36m 39s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 13s		the patch passed
+1 💚	compile	1m 13s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	1m 13s		the patch passed
+1 💚	compile	1m 6s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	1m 6s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 58s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 116 unchanged - 1 fixed = 118 total (was 117)
+1 💚	mvnsite	1m 14s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 36s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 17s		the patch passed
+1 💚	shadedclient	35m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	230m 14s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		369m 52s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestDatanodeManager

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/4/artifact/out/Dockerfile
GITHUB PR	#6915
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 0d928807140a 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 06e85b6
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/4/testReport/
Max. process+thread count	4020 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[zhengchenyu]

Failed unit test: org.apache.hadoop.hdfs.server.blockmanagement.testGetBlockLocationConsiderStorageType is not related, will be fixed in #6919.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	11m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 24s		trunk passed
+1 💚	compile	1m 20s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	1m 17s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	1m 11s		trunk passed
+1 💚	mvnsite	1m 26s		trunk passed
+1 💚	javadoc	1m 8s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	1m 45s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	spotbugs	3m 21s		trunk passed
+1 💚	shadedclient	36m 46s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	37m 8s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
-1 ❌	mvninstall	0m 50s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	compile	0m 54s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt	hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.
-1 ❌	javac	0m 54s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt	hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.
-1 ❌	compile	0m 51s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt	hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.
-1 ❌	javac	0m 51s	/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt	hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 58s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 116 unchanged - 1 fixed = 118 total (was 117)
-1 ❌	mvnsite	0m 53s	/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	1m 41s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
-1 ❌	spotbugs	0m 50s	/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	shadedclient	15m 9s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 54s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		122m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/1/artifact/out/Dockerfile
GITHUB PR	#6915
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 2b521ab8b6c1 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / ea04f95
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/1/testReport/
Max. process+thread count	680 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 15s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	45m 21s		trunk passed
+1 💚	compile	1m 21s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	1m 17s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	1m 10s		trunk passed
+1 💚	mvnsite	1m 27s		trunk passed
+1 💚	javadoc	1m 10s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	1m 45s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	spotbugs	3m 18s		trunk passed
+1 💚	shadedclient	36m 12s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	36m 33s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 11s		the patch passed
+1 💚	compile	1m 15s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javac	1m 15s		the patch passed
+1 💚	compile	1m 7s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	javac	1m 7s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 58s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 116 unchanged - 1 fixed = 118 total (was 117)
+1 💚	mvnsite	1m 18s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	1m 38s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	spotbugs	3m 17s		the patch passed
+1 💚	shadedclient	36m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	218m 12s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		370m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/2/artifact/out/Dockerfile
GITHUB PR	#6915
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 2c8155d72793 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / dd74849
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/2/testReport/
Max. process+thread count	4104 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6915/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[zhengchenyu]

@ayushtkn @Hexiaoqiao @zhangshuyan0 @haiyang1987 @tasanuma

Now that all yetus errors have been fixed, Can you please review this PR?