HDFS-16157. Support configuring DNS record to get list of journal nodes. #3284

LeonGao91 · 2021-08-10T06:34:34Z

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

fengnanli · 2021-08-12T16:07:49Z

LGTM. Let's wait for some time if others have some comments. @Hexiaoqiao FYI. We are doing this in a series to help reducing the dependency on a single host name.

Hexiaoqiao · 2021-08-13T08:54:13Z

Thanks @LeonGao91 and @fengnanli . It is great improvement here. Just some nits here,
a. We should put the new configuration item to hdfs-default.xml.
b. If enable this feature, DNS Resolver has to support resolve multiple hostnames from one domain name, right? Just suggest add some note to HDFSHighAvailabilityWithQJM.md.
Thanks.

LeonGao91 · 2021-08-13T22:01:48Z

thanks for the review @fengnanli @Hexiaoqiao ! will make the change accordingly.

hadoop-yetus · 2021-08-14T07:40:57Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 7s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	markdownlint	0m 1s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 30s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 6s		trunk passed
+1 💚	compile	22m 50s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	19m 33s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 56s		trunk passed
+1 💚	mvnsite	3m 3s		trunk passed
+1 💚	javadoc	2m 6s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 47s		trunk passed
+1 💚	shadedclient	19m 21s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 50s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 17s		the patch passed
+1 💚	compile	22m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 17s		the patch passed
+1 💚	compile	19m 29s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 52s	/results-checkstyle-root.txt	root: The patch generated 3 new + 245 unchanged - 0 fixed = 248 total (was 245)
+1 💚	mvnsite	3m 1s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 4s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 7s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 12s		the patch passed
+1 💚	shadedclient	19m 32s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 57s		hadoop-common in the patch passed.
-1 ❌	unit	360m 51s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		576m 40s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.server.namenode.TestDecommissioningStatus

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/2/artifact/out/Dockerfile
GITHUB PR	#3284
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml markdownlint
uname	Linux 717662c223d8 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0d2242f`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/2/testReport/
Max. process+thread count	2016 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-08-17T03:38:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 0s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	markdownlint	0m 1s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 32s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 4s		trunk passed
+1 💚	compile	22m 47s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	19m 37s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 53s		trunk passed
+1 💚	mvnsite	3m 4s		trunk passed
+1 💚	javadoc	2m 7s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 12s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 50s		trunk passed
+1 💚	shadedclient	19m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 11s		the patch passed
+1 💚	compile	22m 14s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 14s		the patch passed
+1 💚	compile	19m 28s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 49s	/results-checkstyle-root.txt	root: The patch generated 1 new + 245 unchanged - 0 fixed = 246 total (was 245)
+1 💚	mvnsite	3m 1s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 5s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 10s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 8s		the patch passed
+1 💚	shadedclient	19m 25s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 1s		hadoop-common in the patch passed.
-1 ❌	unit	367m 50s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		582m 51s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.server.datanode.TestBlockScanner
	hadoop.hdfs.TestSnapshotCommands
	hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/3/artifact/out/Dockerfile
GITHUB PR	#3284
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml markdownlint
uname	Linux e5548063a24e 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `22d1a56`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/3/testReport/
Max. process+thread count	3143 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-08-17T16:02:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 8s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 25s		trunk passed
+1 💚	compile	23m 7s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	20m 48s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 53s		trunk passed
+1 💚	mvnsite	3m 10s		trunk passed
+1 💚	javadoc	2m 8s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 13s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 51s		trunk passed
+1 💚	shadedclient	19m 25s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s		the patch passed
+1 💚	compile	22m 24s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 24s		the patch passed
+1 💚	compile	21m 1s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	21m 1s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 27s		the patch passed
+1 💚	mvnsite	4m 0s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 39s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 25s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 14s		the patch passed
+1 💚	shadedclient	19m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 14s		hadoop-common in the patch passed.
-1 ❌	unit	364m 0s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		586m 45s

Reason	Tests
Failed junit tests	hadoop.hdfs.tools.TestDFSAdminWithHA

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/4/artifact/out/Dockerfile
GITHUB PR	#3284
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml markdownlint
uname	Linux 9ae600bda287 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `376eba3`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/4/testReport/
Max. process+thread count	3143 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3284/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

LGTM. +1 from my side. The failed unit tests seems not related to changes.

fengnanli · 2021-08-26T00:40:20Z

Thanks for the work @LeonGao91 and review from @Hexiaoqiao

…es contributed by Leon Gao. (apache#3284) * Add DNS resolution for QJM * Add log * Resolve comments * checkstyle * typo

LeonGao91 added 2 commits August 9, 2021 12:21

Add DNS resolution for QJM

8db4ce3

Add log

49f0d2b

fengnanli approved these changes Aug 12, 2021

View reviewed changes

Resolve comments

0d2242f

checkstyle

22d1a56

typo

376eba3

Hexiaoqiao approved these changes Aug 25, 2021

View reviewed changes

fengnanli merged commit b53cae0 into apache:trunk Aug 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-16157. Support configuring DNS record to get list of journal nodes. #3284

HDFS-16157. Support configuring DNS record to get list of journal nodes. #3284

Uh oh!

LeonGao91 commented Aug 10, 2021

Uh oh!

fengnanli commented Aug 12, 2021

Uh oh!

Hexiaoqiao commented Aug 13, 2021

Uh oh!

LeonGao91 commented Aug 13, 2021

Uh oh!

hadoop-yetus commented Aug 14, 2021

Uh oh!

hadoop-yetus commented Aug 17, 2021

Uh oh!

hadoop-yetus commented Aug 17, 2021

Uh oh!

Hexiaoqiao left a comment

Uh oh!

fengnanli commented Aug 26, 2021

Uh oh!

Uh oh!

HDFS-16157. Support configuring DNS record to get list of journal nodes. #3284

HDFS-16157. Support configuring DNS record to get list of journal nodes. #3284

Uh oh!

Conversation

LeonGao91 commented Aug 10, 2021

NOTICE

Uh oh!

fengnanli commented Aug 12, 2021

Uh oh!

Hexiaoqiao commented Aug 13, 2021

Uh oh!

LeonGao91 commented Aug 13, 2021

Uh oh!

hadoop-yetus commented Aug 14, 2021

Uh oh!

hadoop-yetus commented Aug 17, 2021

Uh oh!

hadoop-yetus commented Aug 17, 2021

Uh oh!

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Uh oh!

fengnanli commented Aug 26, 2021

Uh oh!

Uh oh!