HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode #3643

tomscut · 2021-11-11T00:31:47Z

Datanodes already have Read, Write, Sync and Flush metrics. We should add NativeCopy and Transfer as well.

Here is a partial look after the change:

hadoop-yetus · 2021-11-11T08:10:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 8s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 56s		trunk passed
+1 💚	compile	1m 22s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 14s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 22s		trunk passed
+1 💚	javadoc	0m 56s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 22s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 18s		trunk passed
+1 💚	shadedclient	25m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 13s		the patch passed
+1 💚	compile	1m 16s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 16s		the patch passed
+1 💚	compile	1m 8s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 8s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 52s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 37 unchanged - 0 fixed = 39 total (was 37)
+1 💚	mvnsite	1m 13s		the patch passed
+1 💚	javadoc	0m 47s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 18s		the patch passed
+1 💚	shadedclient	25m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	351m 5s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		457m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/1/artifact/out/Dockerfile
GITHUB PR	#3643
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux cb9373ab4042 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / fccd6d1cc3ea25eed9b478de50d4087f1fe0d9d4
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/1/testReport/
Max. process+thread count	1961 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2021-11-11T08:17:38Z

Hi @aajisaka @tasanuma @goiri @jojochuang @ferhui , could you please review the code. Thanks a lot.

hadoop-yetus · 2021-11-11T16:19:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 56s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 41s		trunk passed
+1 💚	compile	1m 46s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 32s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 8s		trunk passed
+1 💚	mvnsite	1m 42s		trunk passed
+1 💚	javadoc	1m 14s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 38s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 41s		trunk passed
+1 💚	shadedclient	26m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	1m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 17s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 53s		the patch passed
+1 💚	mvnsite	1m 13s		the patch passed
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 21s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 18s		the patch passed
+1 💚	shadedclient	24m 56s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	373m 34s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 43s		The patch does not generate ASF License warnings.
		481m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/2/artifact/out/Dockerfile
GITHUB PR	#3643
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux e4e36736f873 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 2a4d7bd608c413d49bd0800cb3652a88e7e822fe
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/2/testReport/
Max. process+thread count	1884 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

ferhui

LGTM

tomscut · 2021-11-12T02:05:29Z

Thanks @ferhui for your review.

tasanuma · 2021-11-12T02:21:57Z

Could you also update hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md?

tomscut · 2021-11-12T02:35:18Z

Could you also update hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md?

Thanks @tasanuma for your suggestion. I will update the document ASAP.

hadoop-yetus · 2021-11-12T05:18:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	patch	0m 19s		#3643 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem	Report/Notes
GITHUB PR	#3643
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/3/console
versions	git=2.17.1
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-12T14:00:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	markdownlint	0m 1s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 45s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 51s		trunk passed
+1 💚	compile	24m 16s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 9s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 48s		trunk passed
+1 💚	mvnsite	3m 9s		trunk passed
+1 💚	javadoc	2m 16s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 9s		trunk passed
+1 💚	shadedclient	22m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 10s		the patch passed
+1 💚	compile	23m 55s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	23m 55s		the patch passed
+1 💚	compile	21m 4s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	21m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 43s		the patch passed
+1 💚	mvnsite	3m 4s		the patch passed
+1 💚	javadoc	2m 14s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 17s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 23s		the patch passed
+1 💚	shadedclient	22m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 47s		hadoop-common in the patch passed.
+1 💚	unit	240m 54s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 9s		The patch does not generate ASF License warnings.
		471m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/4/artifact/out/Dockerfile
GITHUB PR	#3643
Optional Tests	dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle
uname	Linux dc7ff5afd3e9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `a0de2dc`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/4/testReport/
Max. process+thread count	3125 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-12T14:11:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 39s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 50s		trunk passed
+1 💚	compile	24m 31s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 0s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 52s		trunk passed
+1 💚	mvnsite	3m 17s		trunk passed
+1 💚	javadoc	2m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 16s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 56s		trunk passed
+1 💚	shadedclient	22m 55s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 17s		the patch passed
+1 💚	compile	23m 43s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	23m 43s		the patch passed
+1 💚	compile	21m 0s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	21m 0s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	checkstyle	3m 50s		the patch passed
+1 💚	mvnsite	3m 21s		the patch passed
+1 💚	javadoc	2m 15s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 24s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 13s		the patch passed
+1 💚	shadedclient	22m 44s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 52s		hadoop-common in the patch passed.
+1 💚	unit	248m 10s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 14s		The patch does not generate ASF License warnings.
		478m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/5/artifact/out/Dockerfile
GITHUB PR	#3643
Optional Tests	dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle
uname	Linux abc1ce434a8e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `a0de2dc`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/5/testReport/
Max. process+thread count	3283 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

ayushtkn

LGTM in general dropped some minor nits regarding the test code

...s/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java

hadoop-yetus · 2021-11-15T01:25:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	16m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 53s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 23s		trunk passed
+1 💚	compile	21m 43s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 58s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 42s		trunk passed
+1 💚	mvnsite	3m 10s		trunk passed
+1 💚	javadoc	2m 17s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 19s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 42s		trunk passed
+1 💚	shadedclient	22m 38s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 7s		the patch passed
+1 💚	compile	21m 0s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	21m 0s		the patch passed
+1 💚	compile	18m 56s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	18m 56s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 33s		the patch passed
+1 💚	mvnsite	3m 12s		the patch passed
+1 💚	javadoc	2m 16s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 20s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 6s		the patch passed
+1 💚	shadedclient	22m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 21s		hadoop-common in the patch passed.
-1 ❌	unit	376m 9s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 7s		The patch does not generate ASF License warnings.
		610m 38s

Reason	Tests
Failed junit tests	hadoop.hdfs.web.TestWebHdfsFileSystemContract

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/6/artifact/out/Dockerfile
GITHUB PR	#3643
Optional Tests	dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle
uname	Linux af9395139069 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `cdad4d5`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/6/testReport/
Max. process+thread count	2658 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3643/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2021-11-15T01:40:28Z

The failed unit test is unrelated to the change. And work fine locally.

@tasanuma @ayushtkn Please take a look. Thank you very much.

ayushtkn

Changes LGTM.

tomscut · 2021-11-15T04:59:32Z

Changes LGTM.

Thank @ayushtkn for your review.

tasanuma

LGTM.

tasanuma · 2021-11-16T02:19:58Z

Merged. Thanks for your contribution, @tomscut, and thanks for reviewing it, @ferhui and @ayushtkn.

tomscut · 2021-11-16T02:36:00Z

Thanks @tasanuma for the merge.

tasanuma · 2021-11-16T03:21:07Z

@tomscut I'd like to cherry-pick it to lower branches, but there are small conflicts. Could you create another PR for branch-3.3? The variable of name in TestFsDatasetImpl doesn't exist in branch-3.3.

tomscut · 2021-11-16T03:24:02Z

@tomscut I'd like to cherry-pick it to lower branches, but there are small conflicts. Could you create another PR for branch-3.3? The variable of name in TestFsDatasetImpl doesn't exist in branch-3.3.

Thank you for reminding me. I would love to do this and I will submit a separate PR later.

tomscut · 2021-11-16T06:36:47Z

@tomscut I'd like to cherry-pick it to lower branches, but there are small conflicts. Could you create another PR for branch-3.3? The variable of name in TestFsDatasetImpl doesn't exist in branch-3.3.

Hi @tasanuma , I submitted a PR #3666 for branch-3.3. Please help review it after the build. Thank you very much.

…de (apache#3643) Reviewed-by: Hui Fei <ferhui@apache.org> Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>

ferhui approved these changes Nov 12, 2021

View reviewed changes

HDFS-16315. Add metrics related to Transfer and NativeCopy to DataNode

ced25f8

tomscut force-pushed the HDFS-16315 branch from 20ce06b to ced25f8 Compare November 12, 2021 06:07

remove unwanted code

a0de2dc

ayushtkn reviewed Nov 14, 2021

View reviewed changes

...s/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java Outdated Show resolved Hide resolved

...s/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java Outdated Show resolved Hide resolved

optimize the unit test

cdad4d5

tomscut requested a review from ayushtkn November 15, 2021 01:40

ayushtkn approved these changes Nov 15, 2021

View reviewed changes

tasanuma self-requested a review November 15, 2021 16:25

tasanuma approved these changes Nov 16, 2021

View reviewed changes

tasanuma merged commit c9f95b0 into apache:trunk Nov 16, 2021

HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode #3643

HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode #3643

Uh oh!

Conversation

tomscut commented Nov 11, 2021

Uh oh!

hadoop-yetus commented Nov 11, 2021

Uh oh!

tomscut commented Nov 11, 2021

Uh oh!

hadoop-yetus commented Nov 11, 2021

Uh oh!

ferhui left a comment

Choose a reason for hiding this comment

Uh oh!

tomscut commented Nov 12, 2021

Uh oh!

tasanuma commented Nov 12, 2021

Uh oh!

tomscut commented Nov 12, 2021

Uh oh!

hadoop-yetus commented Nov 12, 2021

Uh oh!

hadoop-yetus commented Nov 12, 2021

Uh oh!

hadoop-yetus commented Nov 12, 2021

Uh oh!

ayushtkn left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Nov 15, 2021

Uh oh!

tomscut commented Nov 15, 2021

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

tomscut commented Nov 15, 2021

Uh oh!

tasanuma left a comment

Choose a reason for hiding this comment

Uh oh!

tasanuma commented Nov 16, 2021

Uh oh!

tomscut commented Nov 16, 2021

Uh oh!

tasanuma commented Nov 16, 2021

Uh oh!

tomscut commented Nov 16, 2021

Uh oh!

tomscut commented Nov 16, 2021

Uh oh!

Uh oh!

ayushtkn left a comment •

edited

Loading