HDFS-16350. Datanode start time should be set after RPC server starts successfully

[virajjasani]

Description of PR

We set start time of Datanode when the class is instantiated but it should be ideally set only after RPC server starts and RPC handlers are initialized to serve client requests.

How was this patch tested?

Tested locally. UI screenshot:

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

[virajjasani]

@tomscut @goiri could you please take a look? Thanks

[tomscut]

@tomscut @goiri could you please take a look? Thanks

Thanks @virajjasani for reminding me here. At present, both Router and Namenode obtain startTime in this way. Do you mean there will be a small gap?

[virajjasani]

Thanks @tomscut. Yes this is a minor improvement in the behaviour. Basically, we should set the time according to the actual server startup time. The point is by the time RPC servers are started, Datanode is not considered started so we should keep the start time accordingly, but as I mentioned, it's minor improvement but the timing should be more accurate I believe.

At present, both Router and Namenode obtain startTime in this way.

In that case, they might also require change.

[virajjasani]

Starttime calculation for Namenode and Router seem better than Datanode case, because it's not the main classes that have starttime defined (i.e. NameNode and DFSRouter) but rather other classes that are internally initialized (e.g. FSNamesystem and Router) as part of the main process initialization, have the starttime defined. In the case of Datanode, the main class itself (i.e. DataNode) has starttime defined but it is initialized soon as the Datanode is instantiated. Hence, I think we should only improve Datanode's starttime to match with the actual process start time.

I understand that for a healthy process initialization, there should not be much difference in starttime if we keep it at instance init level (without this patch) vs if we keep it after RPC server is started (with this patch), but my concern is that if for some reason, RPC handler start time is significantly delayed, then we would have incorrect starttime reported.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 55s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 1s		trunk passed
+1 💚	compile	1m 27s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 17s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 26s		trunk passed
+1 💚	javadoc	1m 1s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 30s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 32s		trunk passed
+1 💚	shadedclient	25m 59s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 17s		the patch passed
+1 💚	compile	1m 25s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 25s		the patch passed
+1 💚	compile	1m 14s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 53s		the patch passed
+1 💚	mvnsite	1m 20s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 25s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 21s		the patch passed
+1 💚	shadedclient	25m 6s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	313m 46s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		421m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3711/5/artifact/out/Dockerfile
GITHUB PR	#3711
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux aec6689d0c15 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 776b654
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3711/5/testReport/
Max. process+thread count	2267 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3711/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

[virajjasani]

@ferhui could you please also take a look? Thanks

[ferhui]

LGTM

[ferhui]

@virajjasani Thanks for contribution. @tomscut Thanks for review!