HDFS-16076. Avoid using slow DataNodes for reading by sorting locations #3117

tomscut · 2021-06-17T15:38:16Z

After sorting the expected location list will be: live -> slow -> stale -> staleAndSlow -> entering_maintenance -> decommissioned. This reduces the probability that slow nodes will be used for reading.

hadoop-yetus · 2021-06-17T22:55:06Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 59s		trunk passed
+1 💚	compile	1m 25s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 4s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 56s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 24s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 20s		trunk passed
+1 💚	shadedclient	18m 54s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 14s		the patch passed
+1 💚	compile	1m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 19s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 12s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 59s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 581 unchanged - 0 fixed = 586 total (was 581)
+1 💚	mvnsite	1m 16s		the patch passed
+1 💚	javadoc	0m 50s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 18s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 19s		the patch passed
+1 💚	shadedclient	18m 49s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	343m 36s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		435m 37s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
	hadoop.hdfs.TestHDFSFileSystemContract

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/1/artifact/out/Dockerfile
GITHUB PR	#3117
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 27b49712a184 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 38d29b8d098c8084806396aef4203a1cf7253874
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/1/testReport/
Max. process+thread count	2341 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-06-18T09:15:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	20m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 13s		trunk passed
+1 💚	compile	1m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 2s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 55s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 20s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 17s		trunk passed
+1 💚	shadedclient	18m 49s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 16s		the patch passed
+1 💚	compile	1m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 19s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 57s		the patch passed
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 24s		the patch passed
+1 💚	shadedclient	18m 48s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	360m 44s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 20s		The patch does not generate ASF License warnings.
		473m 22s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/2/artifact/out/Dockerfile
GITHUB PR	#3117
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux e849b04741eb 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / c8d691317eaaa9c5cf0f44aebdd2e76cbec394ac
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/2/testReport/
Max. process+thread count	2318 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-06-18T18:24:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 4s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 41s		trunk passed
+1 💚	compile	1m 28s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 20s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 8s		trunk passed
+1 💚	mvnsite	1m 29s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 24s		trunk passed
+1 💚	shadedclient	22m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 21s		the patch passed
+1 💚	compile	1m 24s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 24s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 12s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 3s		the patch passed
+1 💚	mvnsite	1m 21s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	0m 52s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 26s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 25s		the patch passed
+1 💚	shadedclient	17m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	397m 40s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+0 🆗	asflicense	0m 36s		ASF License check generated no output?
		493m 12s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileAppend2
	hadoop.hdfs.server.namenode.TestFsck
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.TestDecommissionWithBackoffMonitor
	hadoop.hdfs.TestViewDistributedFileSystemContract
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
	hadoop.hdfs.TestSnapshotCommands
	hadoop.hdfs.TestHDFSFileSystemContract
	hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks
	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.TestHdfsAdmin
	hadoop.hdfs.TestReadStripedFileWithDNFailure

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/3/artifact/out/Dockerfile
GITHUB PR	#3117
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux ed1c898952eb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / d148bd35d76ef8b790202c1044ef59b777256a5a
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/3/testReport/
Max. process+thread count	2761 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2021-06-19T04:09:16Z

Rebased to the latest commit.

hadoop-yetus · 2021-06-19T12:05:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 13s		trunk passed
+1 💚	compile	1m 26s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 5s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 25s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 17s		trunk passed
+1 💚	shadedclient	18m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 16s		the patch passed
+1 💚	compile	1m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 19s		the patch passed
+1 💚	compile	1m 11s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 11s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 59s		the patch passed
+1 💚	mvnsite	1m 17s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 21s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 21s		the patch passed
+1 💚	shadedclient	18m 45s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	384m 59s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		477m 36s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.web.TestWebHdfsFileSystemContract
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.TestSnapshotCommands
	hadoop.hdfs.server.mover.TestMover
	hadoop.hdfs.TestLeaseRecoveryStriped
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.hdfs.TestInjectionForSimulatedStorage
	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.TestWriteReadStripedFile
	hadoop.hdfs.tools.TestDebugAdmin

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/4/artifact/out/Dockerfile
GITHUB PR	#3117
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux 87883f156701 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6af8a14`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/4/testReport/
Max. process+thread count	2238 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2021-06-21T02:02:41Z

Those failed unit tests work fine locally.

tomscut · 2021-06-21T02:05:47Z

Hi @tasanuma @jojochuang @goiri @Hexiaoqiao @ayushtkn , could you please help to review the code? Thanks.

tasanuma

@tomscut Seems pretty good. I left some very minor comments. The others look good to me.

...p-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSortLocatedBlock.java

hadoop-yetus · 2021-06-23T08:43:06Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 45s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 7s		trunk passed
+1 💚	compile	1m 26s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 4s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 55s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 24s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 14s		trunk passed
+1 💚	shadedclient	18m 53s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	1m 18s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 18s		the patch passed
+1 💚	compile	1m 10s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 57s		the patch passed
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 18s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 22s		the patch passed
+1 💚	shadedclient	18m 43s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	346m 40s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 4s		The patch does not generate ASF License warnings.
		438m 51s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.server.mover.TestMover
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/5/artifact/out/Dockerfile
GITHUB PR	#3117
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux 28e07ef0078f 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `74bb4aa`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/5/testReport/
Max. process+thread count	2328 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3117/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2021-06-23T09:28:48Z

Thanks @tasanuma for your second review. These failed UTs work fine locally.

tasanuma · 2021-06-24T02:28:01Z

Merged it. Thanks for your contribution, @tomscut.

tomscut · 2021-06-24T02:29:53Z

Merged it. Thanks for your contribution, @tomscut.

Thanks @tasanuma .

…ns (#3117) (cherry picked from commit fdef2b4)

…ns (apache#3117)

tomscut mentioned this pull request Jun 19, 2021

HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli… #3119

Merged

HDFS-16076. Avoid using slow DataNodes for reading by sorting locations

6af8a14

tomscut force-pushed the HDFS-16076 branch from d148bd3 to 6af8a14 Compare June 19, 2021 04:06

tasanuma self-requested a review June 22, 2021 16:37

tasanuma reviewed Jun 22, 2021

View reviewed changes

...p-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSortLocatedBlock.java Outdated Show resolved Hide resolved

...p-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSortLocatedBlock.java Outdated Show resolved Hide resolved

remove useless code

74bb4aa

tasanuma approved these changes Jun 23, 2021

View reviewed changes

tasanuma merged commit fdef2b4 into apache:trunk Jun 24, 2021

tasanuma pushed a commit that referenced this pull request Jun 24, 2021

HDFS-16076. Avoid using slow DataNodes for reading by sorting locatio…

5fc61d2

…ns (#3117) (cherry picked from commit fdef2b4)

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

HDFS-16076. Avoid using slow DataNodes for reading by sorting locatio…

3129f61

…ns (apache#3117)

HDFS-16076. Avoid using slow DataNodes for reading by sorting locations #3117

HDFS-16076. Avoid using slow DataNodes for reading by sorting locations #3117

Uh oh!

Conversation

tomscut commented Jun 17, 2021

Uh oh!

hadoop-yetus commented Jun 17, 2021

Uh oh!

hadoop-yetus commented Jun 18, 2021

Uh oh!

hadoop-yetus commented Jun 18, 2021

Uh oh!

tomscut commented Jun 19, 2021

Uh oh!

hadoop-yetus commented Jun 19, 2021

Uh oh!

tomscut commented Jun 21, 2021

Uh oh!

tomscut commented Jun 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tasanuma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Jun 23, 2021

Uh oh!

tomscut commented Jun 23, 2021

Uh oh!

tasanuma commented Jun 24, 2021

Uh oh!

tomscut commented Jun 24, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tomscut commented Jun 21, 2021 •

edited

Loading