HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance #4567

lfxy · 2022-07-16T09:24:14Z

In HDFS-16613, increase the value of dfs.namenode.replication.max-streams-hard-limit would maximize the IO performance of the decommissioning DN, which has a lot of EC blocks. Besides this, we also need to decrease the value of dfs.namenode.reconstruction.pending.timeout-sec, default is 5 minutes, to shorten the interval time for checking pendingReconstructions. Or the decommissioning node would be idle to wait for copy tasks in most of this 5 minutes.

In decommission progress, we may need to reconfigure these 2 parameters several times. In HDFS-14560, the dfs.namenode.replication.max-streams-hard-limit can already be reconfigured dynamically without namenode restart. And the dfs.namenode.reconstruction.pending.timeout-sec parameter also need to be reconfigured dynamically.

hadoop-yetus · 2022-07-16T15:14:48Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 29s		trunk passed
+1 💚	compile	1m 42s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	1m 35s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 26s		trunk passed
+1 💚	mvnsite	1m 48s		trunk passed
+1 💚	javadoc	1m 25s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 41s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 45s		trunk passed
+1 💚	shadedclient	22m 52s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 22s		the patch passed
+1 💚	compile	1m 26s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	1m 26s		the patch passed
+1 💚	compile	1m 20s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 4s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 160 unchanged - 0 fixed = 162 total (was 160)
+1 💚	mvnsite	1m 25s		the patch passed
+1 💚	javadoc	0m 56s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 29s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 24s		the patch passed
+1 💚	shadedclient	22m 45s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	237m 38s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 16s		The patch does not generate ASF License warnings.
		348m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4567/1/artifact/out/Dockerfile
GITHUB PR	#4567
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 95af13760eec 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / ebf086b58b2284acdd8f1003b07b0a87da86edd2
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4567/1/testReport/
Max. process+thread count	2839 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4567/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-07-16T21:06:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	37m 34s		trunk passed
+1 💚	compile	1m 43s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	1m 38s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 27s		trunk passed
+1 💚	mvnsite	1m 45s		trunk passed
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 49s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 44s		trunk passed
+1 💚	shadedclient	22m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 27s		the patch passed
+1 💚	compile	1m 25s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	1m 25s		the patch passed
+1 💚	compile	1m 19s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 19s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 3s		the patch passed
+1 💚	mvnsite	1m 27s		the patch passed
+1 💚	javadoc	0m 59s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 34s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 21s		the patch passed
+1 💚	shadedclient	22m 38s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	235m 52s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 12s		The patch does not generate ASF License warnings.
		345m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4567/2/artifact/out/Dockerfile
GITHUB PR	#4567
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 00885af5d0a2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `bdd9121`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4567/2/testReport/
Max. process+thread count	3287 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4567/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hi-adachi

+1 (non-binding)

lfxy · 2022-08-09T15:56:10Z

@jojochuang Could you help to review and merge this patch if you have time? Thank you.

tasanuma

@lfxy Thanks for the PR. The change looks good to me.
Sorry I couldn't reply to your PR for a long time.

tasanuma · 2022-09-12T02:45:30Z

Merged it. Thanks for your contribution, @lfxy. Thanks for your review, @hi-adachi.

tasanuma · 2022-09-13T09:37:43Z

I forgot to add the ticket number to the commit message. Sorry.

…4567) Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>

…freshable (apache#4567) Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>

…ble (apache#4567) Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>

Allow block reconstruction pending timeout to be refreshable

bdd9121

lfxy force-pushed the feature/HDFS-16663 branch from ebf086b to bdd9121 Compare July 16, 2022 15:19

hi-adachi approved these changes Jul 25, 2022

View reviewed changes

tasanuma approved these changes Sep 12, 2022

View reviewed changes

tasanuma merged commit 1923096 into apache:trunk Sep 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance #4567

HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance #4567

lfxy commented Jul 16, 2022 •

edited

Loading

hadoop-yetus commented Jul 16, 2022

hadoop-yetus commented Jul 16, 2022

hi-adachi left a comment

lfxy commented Aug 9, 2022

tasanuma left a comment

tasanuma commented Sep 12, 2022

tasanuma commented Sep 13, 2022

HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance #4567

HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance #4567

Conversation

lfxy commented Jul 16, 2022 • edited Loading

hadoop-yetus commented Jul 16, 2022

hadoop-yetus commented Jul 16, 2022

hi-adachi left a comment

Choose a reason for hiding this comment

lfxy commented Aug 9, 2022

tasanuma left a comment

Choose a reason for hiding this comment

tasanuma commented Sep 12, 2022

tasanuma commented Sep 13, 2022

lfxy commented Jul 16, 2022 •

edited

Loading