HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta… #574

infraio · 2019-09-02T07:33:19Z

…rtFailoverSplitWithoutZk

Apache-HBase · 2019-09-02T17:44:22Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	47	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1	mvninstall	418	master passed
+1	compile	69	master passed
+1	checkstyle	99	master passed
+1	shadedjars	350	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	42	master passed
0	spotbugs	282	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	278	master passed
		_ Patch Compile Tests _
+1	mvninstall	383	the patch passed
+1	compile	71	the patch passed
+1	javac	71	the patch passed
-1	checkstyle	95	hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	323	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	1165	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	36	the patch passed
+1	findbugs	257	the patch passed
		_ Other Tests _
-1	unit	10357	hbase-server in the patch failed.
+1	asflicense	30	The patch does not generate ASF License warnings.
		14436

Reason	Tests
Failed junit tests	hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/1/artifact/out/Dockerfile
GITHUB PR	#574
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux f0ef73ea245b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-574/out/precommit/personality/provided.sh
git revision	master / `8e8bd8b`
Default Java	1.8.0_181
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/1/artifact/out/diff-checkstyle-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/1/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/1/testReport/
Max. process+thread count	4897 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/1/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Apache9 · 2019-09-03T00:26:14Z

We need to know what is the purpose of the test. The current fix looks like we just give up testing if some conditions are not match...

infraio · 2019-09-03T00:39:08Z

We need to know what is the purpose of the test. The current fix looks like we just give up testing if some conditions are not match...

It was added by HBASE-21565: Delete dead server from dead server list too early leads to concurrent Server Crash Procedures(SCP) for a same server. So the test purpose is assertFalse(UTIL.getHBaseCluster().getMaster().getServerManager().expireServer(testServer)). It should not submit SCP again if there arleady had a SCP for this server.

infraio · 2019-09-03T01:42:47Z

Updated the patch. Will test the submit SCP again when serverNode is null.

Apache9

I still not fully understand this UT here. I guess we need to find a way to make sure that the serverNode is not null?

Apache9 · 2019-09-03T13:13:31Z

hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestClusterRestartFailover.java

+      LOG.info("start to find the procedure of SCP for the severName we choose");
+      Procedure<?> procedure = UTIL.getHBaseCluster().getMaster().getProcedures().stream().filter(
+          p -> (p instanceof ServerCrashProcedure) &&
+              ((ServerCrashProcedure) p).getServerName().equals(testServer)).findAny().get();


Optional.get will never return null, it will throw a NoSuchElementException...

infraio · 2019-09-03T13:59:43Z

I still not fully understand this UT here. I guess we need to find a way to make sure that the serverNode is not null?

Checked HBASE-21565. I thought this ut is added to test should submit SCP for a same server twice. Need to mock SCP to stuck it if make sure that serverNode is not null.

Apache9 · 2019-09-03T14:23:50Z

Checked HBASE-21565. I thought this ut is added to test should submit SCP for a same server twice. Need to mock SCP to stuck it if make sure that serverNode is not null.

Yes, this is possible. You can see TestSCPGetRegionsRace, where we override AssignmentManager and hang the SCP when it calls getRegionsOnServer.

Apache-HBase · 2019-09-03T22:08:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	39	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1	mvninstall	326	master passed
+1	compile	57	master passed
+1	checkstyle	78	master passed
+1	shadedjars	271	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	38	master passed
0	spotbugs	240	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	239	master passed
		_ Patch Compile Tests _
+1	mvninstall	293	the patch passed
+1	compile	54	the patch passed
+1	javac	54	the patch passed
-1	checkstyle	74	hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	272	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	937	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	33	the patch passed
+1	findbugs	244	the patch passed
		_ Other Tests _
+1	unit	9396	hbase-server in the patch passed.
+1	asflicense	35	The patch does not generate ASF License warnings.
		12757

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/2/artifact/out/Dockerfile
GITHUB PR	#574
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 184d0bd1364d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-574/out/precommit/personality/provided.sh
git revision	master / `e50ec76`
Default Java	1.8.0_181
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/2/artifact/out/diff-checkstyle-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/2/testReport/
Max. process+thread count	4374 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/2/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-09-05T06:44:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	0m 0s	Docker mode activated.
💔	patch	2m 14s	#574 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/3/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-09-06T06:10:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	6m 54s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	6m 9s	master passed
💚	compile	1m 9s	master passed
💚	checkstyle	1m 46s	master passed
💚	shadedjars	5m 24s	branch has no errors when building our shaded downstream artifacts.
💚	javadoc	0m 37s	master passed
💙	spotbugs	4m 50s	Used deprecated FindBugs config; considering switching to SpotBugs.
💚	findbugs	4m 47s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 56s	the patch passed
💚	compile	0m 57s	the patch passed
💚	javac	0m 57s	the patch passed
💔	checkstyle	1m 39s	hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	shadedjars	5m 39s	patch has no errors when building our shaded downstream artifacts.
💚	hadoopcheck	22m 50s	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚	javadoc	0m 45s	the patch passed
💚	findbugs	4m 47s	the patch passed
		_ Other Tests _
💔	unit	266m 22s	hbase-server in the patch failed.
💚	asflicense	0m 30s	The patch does not generate ASF License warnings.
		345m 24s

Subsystem	Report/Notes
Docker	Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/4/artifact/out/Dockerfile
GITHUB PR	#574
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 8689ff1d52c4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-574/out/precommit/personality/provided.sh
git revision	master / `fbd5b5e`
Default Java	1.8.0_181
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/4/artifact/out/diff-checkstyle-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/4/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/4/testReport/
Max. process+thread count	4683 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/4/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-09-06T17:35:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	0m 30s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	6m 0s	master passed
💚	compile	1m 2s	master passed
💚	checkstyle	1m 33s	master passed
💚	shadedjars	5m 8s	branch has no errors when building our shaded downstream artifacts.
💚	javadoc	0m 39s	master passed
💙	spotbugs	4m 44s	Used deprecated FindBugs config; considering switching to SpotBugs.
💚	findbugs	4m 42s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 29s	the patch passed
💚	compile	1m 0s	the patch passed
💚	javac	1m 0s	the patch passed
💚	checkstyle	1m 29s	the patch passed
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	shadedjars	5m 4s	patch has no errors when building our shaded downstream artifacts.
💚	hadoopcheck	20m 11s	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚	javadoc	0m 47s	the patch passed
💚	findbugs	5m 32s	the patch passed
		_ Other Tests _
💚	unit	176m 11s	hbase-server in the patch passed.
💚	asflicense	0m 26s	The patch does not generate ASF License warnings.
		243m 26s

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/5/artifact/out/Dockerfile
GITHUB PR	#574
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 8c5cd2703dc1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-574/out/precommit/personality/provided.sh
git revision	master / `51b6ce3`
Default Java	1.8.0_181
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/5/testReport/
Max. process+thread count	4457 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/5/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Apache9

+1.

Apache-HBase · 2019-09-07T19:59:38Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	1m 6s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 1s	The patch appears to include 2 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	5m 59s	master passed
💚	compile	0m 58s	master passed
💚	checkstyle	1m 32s	master passed
💚	shadedjars	4m 55s	branch has no errors when building our shaded downstream artifacts.
💚	javadoc	0m 39s	master passed
💙	spotbugs	4m 30s	Used deprecated FindBugs config; considering switching to SpotBugs.
💚	findbugs	4m 27s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 21s	the patch passed
💚	compile	0m 58s	the patch passed
💚	javac	0m 58s	the patch passed
💚	checkstyle	1m 32s	the patch passed
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	shadedjars	4m 55s	patch has no errors when building our shaded downstream artifacts.
💚	hadoopcheck	17m 10s	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚	javadoc	0m 34s	the patch passed
💚	findbugs	4m 35s	the patch passed
		_ Other Tests _
💔	unit	232m 20s	hbase-server in the patch failed.
💚	asflicense	0m 27s	The patch does not generate ASF License warnings.
		294m 23s

Subsystem	Report/Notes
Docker	Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/6/artifact/out/Dockerfile
GITHUB PR	#574
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 2b856c5c70f7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-574/out/precommit/personality/provided.sh
git revision	master / `78bae9e`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/6/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/6/testReport/
Max. process+thread count	4785 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-574/6/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

…rtFailoverSplitWithoutZk

…rtFailoverSplitWithoutZk (#574) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…rtFailoverSplitWithoutZk (apache#574) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…rtFailoverSplitWithoutZk (apache#574) Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 5e98129) Change-Id: Ib1f0549c24d2b81559e33bc22d4096b246a1e37a

infraio force-pushed the HBASE-22964 branch from b74b4f4 to fd73931 Compare September 3, 2019 01:41

Apache9 requested changes Sep 3, 2019

View reviewed changes

infraio force-pushed the HBASE-22964 branch from fd73931 to 43a9f2b Compare September 5, 2019 06:38

infraio force-pushed the HBASE-22964 branch from 43a9f2b to 81eb6b6 Compare September 6, 2019 00:15

infraio force-pushed the HBASE-22964 branch from 81eb6b6 to 58b6bdb Compare September 6, 2019 13:11

Apache9 approved these changes Sep 7, 2019

View reviewed changes

infraio force-pushed the HBASE-22964 branch from 58b6bdb to 6cd98de Compare September 7, 2019 14:54

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta…

6cd98de

…rtFailoverSplitWithoutZk

infraio merged commit d8e5c87 into apache:master Sep 8, 2019

infraio deleted the HBASE-22964 branch September 8, 2019 10:47

asfgit pushed a commit that referenced this pull request Sep 8, 2019

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta…

a4a34e0

…rtFailoverSplitWithoutZk (#574) Signed-off-by: Duo Zhang <zhangduo@apache.org>

asfgit pushed a commit that referenced this pull request Sep 9, 2019

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta…

5e98129

…rtFailoverSplitWithoutZk (#574) Signed-off-by: Duo Zhang <zhangduo@apache.org>

infraio added a commit to infraio/hbase that referenced this pull request Aug 17, 2020

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta…

df39a58

…rtFailoverSplitWithoutZk (apache#574) Signed-off-by: Duo Zhang <zhangduo@apache.org>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta… #574

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta… #574

Uh oh!

infraio commented Sep 2, 2019

Uh oh!

Apache-HBase commented Sep 2, 2019

Uh oh!

Apache9 commented Sep 3, 2019

Uh oh!

infraio commented Sep 3, 2019

Uh oh!

infraio commented Sep 3, 2019

Uh oh!

Apache9 left a comment

Uh oh!

Apache9 Sep 3, 2019

Uh oh!

infraio commented Sep 3, 2019

Uh oh!

Apache9 commented Sep 3, 2019

Uh oh!

Apache-HBase commented Sep 3, 2019

Uh oh!

Apache-HBase commented Sep 5, 2019

Uh oh!

Apache-HBase commented Sep 6, 2019

Uh oh!

Apache-HBase commented Sep 6, 2019

Uh oh!

Apache9 left a comment

Uh oh!

Apache-HBase commented Sep 7, 2019

Uh oh!

Uh oh!

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta… #574

HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterResta… #574

Uh oh!

Conversation

infraio commented Sep 2, 2019

Uh oh!

Apache-HBase commented Sep 2, 2019

Uh oh!

Apache9 commented Sep 3, 2019

Uh oh!

infraio commented Sep 3, 2019

Uh oh!

infraio commented Sep 3, 2019

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

Apache9 Sep 3, 2019

Choose a reason for hiding this comment

Uh oh!

infraio commented Sep 3, 2019

Uh oh!

Apache9 commented Sep 3, 2019

Uh oh!

Apache-HBase commented Sep 3, 2019

Uh oh!

Apache-HBase commented Sep 5, 2019

Uh oh!

Apache-HBase commented Sep 6, 2019

Uh oh!

Apache-HBase commented Sep 6, 2019

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Sep 7, 2019

Uh oh!

Uh oh!