HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. #1048

bharatviswa504 · 2019-07-02T18:57:56Z

No description provided.

anuengineer

I am +1; on this change. But I am wondering if we should add some shutdown handling ..
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html
if I remember correctly, in the Hadoop Executor services helper classes we have shutdown function.

hadoop-yetus · 2019-07-02T20:54:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	75	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	489	trunk passed
+1	compile	263	trunk passed
+1	checkstyle	69	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	954	branch has no errors when building and testing our client artifacts.
+1	javadoc	168	trunk passed
0	spotbugs	355	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	581	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	472	the patch passed
+1	compile	274	the patch passed
+1	javac	274	the patch passed
+1	checkstyle	80	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	746	patch has no errors when building and testing our client artifacts.
+1	javadoc	167	the patch passed
+1	findbugs	584	the patch passed
		_ Other Tests _
+1	unit	320	hadoop-hdds in the patch passed.
-1	unit	1472	hadoop-ozone in the patch failed.
+1	asflicense	47	The patch does not generate ASF License warnings.
		6929

Reason	Tests
Failed junit tests	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
	hadoop.ozone.client.rpc.TestBCSID
	hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerByPipeline
	hadoop.ozone.client.rpc.TestCommitWatcher
	hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
	hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException

Subsystem	Report/Notes
Docker	Client=18.09.5 Server=18.09.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/1/artifact/out/Dockerfile
GITHUB PR	#1048
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 0a50f47d0fd3 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `564758a`
Default Java	1.8.0_212
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/1/testReport/
Max. process+thread count	5284 (vs. ulimit of 5500)
modules	C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-07-02T21:02:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	68	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	468	trunk passed
+1	compile	231	trunk passed
+1	checkstyle	70	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	880	branch has no errors when building and testing our client artifacts.
+1	javadoc	157	trunk passed
0	spotbugs	305	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	494	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	430	the patch passed
+1	compile	253	the patch passed
+1	javac	253	the patch passed
+1	checkstyle	74	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	730	patch has no errors when building and testing our client artifacts.
+1	javadoc	179	the patch passed
+1	findbugs	605	the patch passed
		_ Other Tests _
+1	unit	335	hadoop-hdds in the patch passed.
-1	unit	2108	hadoop-ozone in the patch failed.
+1	asflicense	52	The patch does not generate ASF License warnings.
		7316

Reason	Tests
Failed junit tests	hadoop.ozone.TestMiniOzoneCluster
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
	hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException
	hadoop.ozone.om.TestOzoneManager
	hadoop.ozone.client.rpc.TestContainerStateMachine
	hadoop.ozone.client.rpc.TestCommitWatcher

Subsystem	Report/Notes
Docker	Client=18.09.5 Server=18.09.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/2/artifact/out/Dockerfile
GITHUB PR	#1048
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 7ca884e2b65d 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `564758a`
Default Java	1.8.0_212
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/2/testReport/
Max. process+thread count	3769 (vs. ulimit of 5500)
modules	C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

bharatviswa504 · 2019-07-02T21:17:46Z

Thank You @anuengineer for the review.
As suggested, used HadoopExecutor API, and also invoked shutdown during the stop.

anuengineer · 2019-07-02T22:06:31Z

Please feel free to commit when all checks are complete. Thanks for the patch. Appreciate it.

hadoop-yetus · 2019-07-02T22:58:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	67	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	455	trunk passed
+1	compile	246	trunk passed
+1	checkstyle	60	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	790	branch has no errors when building and testing our client artifacts.
+1	javadoc	159	trunk passed
0	spotbugs	318	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	506	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	428	the patch passed
+1	compile	249	the patch passed
+1	javac	249	the patch passed
+1	checkstyle	80	the patch passed
+1	mvnsite	1	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	646	patch has no errors when building and testing our client artifacts.
+1	javadoc	161	the patch passed
+1	findbugs	521	the patch passed
		_ Other Tests _
+1	unit	241	hadoop-hdds in the patch passed.
-1	unit	1183	hadoop-ozone in the patch failed.
+1	asflicense	49	The patch does not generate ASF License warnings.
		6061

Reason	Tests
Failed junit tests	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
	hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/3/artifact/out/Dockerfile
GITHUB PR	#1048
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 8a5976ede4f7 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 75b1e45
Default Java	1.8.0_212
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/3/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/3/testReport/
Max. process+thread count	5018 (vs. ulimit of 5500)
modules	C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1048/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

)

…e refactor (apache#1048) * SAMZA-2215 : StartpointManager fix for previous CoordinatorStreamStore refactor * Optimize imports on ContainerLaunchUtil

)

bharatviswa504 assigned arp7 and bharatviswa504 Jul 2, 2019

bharatviswa504 requested a review from hanishakoneru July 2, 2019 18:58

bharatviswa504 unassigned arp7 Jul 2, 2019

bharatviswa504 requested a review from arp7 July 2, 2019 18:58

bharatviswa504 added the ozone label Jul 2, 2019

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine.

9b96ddc

bharatviswa504 force-pushed the HDDS-1757 branch from fc8d352 to 9b96ddc Compare July 2, 2019 18:59

anuengineer reviewed Jul 2, 2019

View reviewed changes

fix anu comments.

19b5b7d

anuengineer approved these changes Jul 2, 2019

View reviewed changes

anuengineer merged commit 91cc197 into apache:trunk Jul 2, 2019

elek mentioned this pull request Jul 4, 2019

HDDS-1764. Fix hidden errors in acceptance tests #1059

Closed

bshashikant pushed a commit to bshashikant/hadoop that referenced this pull request Jul 10, 2019

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. (apache#1048

0f1cc4d

)

amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. (apache#1048

4593089

)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. #1048

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. #1048

Uh oh!

bharatviswa504 commented Jul 2, 2019

Uh oh!

anuengineer left a comment

Uh oh!

hadoop-yetus commented Jul 2, 2019

Uh oh!

hadoop-yetus commented Jul 2, 2019

Uh oh!

bharatviswa504 commented Jul 2, 2019

Uh oh!

anuengineer commented Jul 2, 2019

Uh oh!

hadoop-yetus commented Jul 2, 2019

Uh oh!

Uh oh!

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. #1048

HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. #1048

Uh oh!

Conversation

bharatviswa504 commented Jul 2, 2019

Uh oh!

anuengineer left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jul 2, 2019

Uh oh!

hadoop-yetus commented Jul 2, 2019

Uh oh!

bharatviswa504 commented Jul 2, 2019

Uh oh!

anuengineer commented Jul 2, 2019

Uh oh!

hadoop-yetus commented Jul 2, 2019

Uh oh!

Uh oh!