HBASE-23123 Merge_region fails from shell #690

karthikhw · 2019-10-05T02:08:25Z

The deprecated method Admin#mergeRegions is removed in HBase 3.0 but we need to update new API in admin script.

Apache-HBase · 2019-10-05T02:30:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	@author	0m 0s	The patch does not contain any @author tags.
💛	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
💚	mvninstall	5m 29s	master passed
💚	javadoc	0m 16s	master passed
		_ Patch Compile Tests _
💚	mvninstall	4m 52s	the patch passed
💔	rubocop	0m 8s	The patch generated 4 new + 307 unchanged - 0 fixed = 311 total (was 307)
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	javadoc	0m 14s	the patch passed
		_ Other Tests _
💚	unit	7m 10s	hbase-shell in the patch passed.
💚	asflicense	0m 14s	The patch does not generate ASF License warnings.
		20m 7s

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/1/artifact/out/Dockerfile
GITHUB PR	#690
Optional Tests	dupname asflicense javac javadoc unit rubocop
uname	Linux 718bee3c1d11 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-690/out/precommit/personality/provided.sh
git revision	master / `f69e770`
Default Java	1.8.0_181
rubocop	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/1/artifact/out/diff-patch-rubocop.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/1/testReport/
Max. process+thread count	2684 (vs. ulimit of 10000)
modules	C: hbase-shell U: hbase-shell
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/1/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.75.0
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

karthikhw · 2019-10-05T08:03:13Z

Thank you @saintstack for the approval.

Apache-HBase · 2019-10-05T08:33:10Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	1m 11s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	@author	0m 0s	The patch does not contain any @author tags.
💛	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
💚	mvninstall	6m 4s	master passed
💚	javadoc	0m 14s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 29s	the patch passed
💔	rubocop	0m 7s	The patch generated 1 new + 307 unchanged - 0 fixed = 308 total (was 307)
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	javadoc	0m 12s	the patch passed
		_ Other Tests _
💚	unit	8m 25s	hbase-shell in the patch passed.
💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		23m 4s

Subsystem	Report/Notes
Docker	Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/2/artifact/out/Dockerfile
GITHUB PR	#690
Optional Tests	dupname asflicense javac javadoc unit rubocop
uname	Linux d9bb3a44f762 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-690/out/precommit/personality/provided.sh
git revision	master / `f69e770`
Default Java	1.8.0_181
rubocop	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/2/artifact/out/diff-patch-rubocop.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/2/testReport/
Max. process+thread count	2610 (vs. ulimit of 10000)
modules	C: hbase-shell U: hbase-shell
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/2/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.75.0
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2019-10-06T05:59:02Z

We do not have a UT in hbase-shell for this command?

karthikhw · 2019-10-06T18:16:39Z

@Apache9 @virajjasani
It looks to me no test method defined for merge_region. I see only few test from admin_test.rb but not all shell commands included in that test. I think that's our gap. I just ran admin_test.rb test but somehow it is failing. Let me check.

However, I am not sure if maven skip running this test ?? When I run "mvn test" on "hbase-shell" module, it only runs two test classes and reports success but I don't find all of them are running. This is weird. I will check on this today.

[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.client.TestShellNoCluster
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.622 s - in org.apache.hadoop.hbase.client.TestShellNoCluster
[INFO] Running org.apache.hadoop.hbase.client.TestTableShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.476 s - in org.apache.hadoop.hbase.client.TestTableShell
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

@virajjasani "return statement is redundant" is a warning of ruby check style. This is about I return "nil" without any variable check, something like "if (var > x) else return nil" etc.

I directly return nil in script because the method mergeRegionsAsync returns void.

Future<Void> mergeRegionsAsync(byte[] nameOfRegionA, byte[] nameOfRegionB,
boolean forcible) throws IOException {

If I don't use nil, then I will get CompletableFuture output.
=> #Java::JavaUtilConcurrent::CompletableFuture:0x68631b1d

Example:
hbase(main):001:0> merge_region '39ac83efe7b96b875d0a9e63313a3717','ff68d30015dc2565eaea1a16207e2e91'
Took 0.9494 seconds
=> #Java::JavaUtilConcurrent::CompletableFuture:0x68631b1d

virajjasani · 2019-10-06T18:27:30Z

I directly return nil in script because the method mergeRegionsAsync returns void.

Correct, just realized by default async method's return value will be passed on to merge_region.rb.

Regarding the tests, better to include test for merge_region() with this PR. And I agree that "mvn test" doesn't run all tests for hbase-shell and we have to manually run them as of now.

karthikhw · 2019-10-06T18:50:49Z

Great @virajjasani I will add merge_region testcase. I am very curious that why would we need to run hbase-shell test cases manually, Sorry. Is it because the test category is LargeTest?
I see admin_test.rb should run as part of TestAdminShell.

karthikhw · 2019-10-06T19:06:27Z

I got @virajjasani. I see TestAdminShell runs here.

karthikhw · 2019-10-07T01:34:36Z

@Apache9 @virajjasani I included test case for merge_region.
By the way, my test case for TestAdminShell was failing on my MAC is due to Openfilelimit. The test mechanism is super intelligent :) It runs successfully after increasing file descriptor limit.

Thread LEAK? -, OpenFileDescriptor=744 (was 743) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=10240 (was 10240), SystemLoadAverage=234 (was 274), ProcessCount=390 (was 390), AvailableMemoryMB=620 (was 647)

Apache-HBase · 2019-10-07T02:01:03Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	5m 52s	master passed
💚	javadoc	0m 16s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 9s	the patch passed
💔	rubocop	0m 14s	The patch generated 1 new + 471 unchanged - 0 fixed = 472 total (was 471)
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	javadoc	0m 13s	the patch passed
		_ Other Tests _
💚	unit	7m 11s	hbase-shell in the patch passed.
💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		20m 56s

Subsystem	Report/Notes
Docker	Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/3/artifact/out/Dockerfile
GITHUB PR	#690
Optional Tests	dupname asflicense javac javadoc unit rubocop
uname	Linux 26e8372c1e81 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-690/out/precommit/personality/provided.sh
git revision	master / `2144690`
Default Java	1.8.0_181
rubocop	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/3/artifact/out/diff-patch-rubocop.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/3/testReport/
Max. process+thread count	2744 (vs. ulimit of 10000)
modules	C: hbase-shell U: hbase-shell
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-690/3/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.75.0
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

HBASE-23123 Merge_region fails from shell

f53a1ce

saintstack approved these changes Oct 5, 2019

View reviewed changes

Checkstyle fix except for Redundant return check

ce1d670

HBASE-23123 - Added merge_region test case

087891d

HBASE-23123 testcase refactors

ca8af44

virajjasani approved these changes Oct 7, 2019

View reviewed changes

saintstack merged commit ba12d5b into apache:master Oct 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-23123 Merge_region fails from shell #690

HBASE-23123 Merge_region fails from shell #690

Uh oh!

karthikhw commented Oct 5, 2019

Uh oh!

Apache-HBase commented Oct 5, 2019

Uh oh!

karthikhw commented Oct 5, 2019

Uh oh!

Apache-HBase commented Oct 5, 2019

Uh oh!

Apache9 commented Oct 6, 2019

Uh oh!

karthikhw commented Oct 6, 2019 •

edited

Loading

Uh oh!

virajjasani commented Oct 6, 2019 •

edited

Loading

Uh oh!

karthikhw commented Oct 6, 2019

Uh oh!

karthikhw commented Oct 6, 2019

Uh oh!

karthikhw commented Oct 7, 2019

Uh oh!

Apache-HBase commented Oct 7, 2019

Uh oh!

Uh oh!

HBASE-23123 Merge_region fails from shell #690

HBASE-23123 Merge_region fails from shell #690

Uh oh!

Conversation

karthikhw commented Oct 5, 2019

Uh oh!

Apache-HBase commented Oct 5, 2019

Uh oh!

karthikhw commented Oct 5, 2019

Uh oh!

Apache-HBase commented Oct 5, 2019

Uh oh!

Apache9 commented Oct 6, 2019

Uh oh!

karthikhw commented Oct 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

virajjasani commented Oct 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

karthikhw commented Oct 6, 2019

Uh oh!

karthikhw commented Oct 6, 2019

Uh oh!

karthikhw commented Oct 7, 2019

Uh oh!

Apache-HBase commented Oct 7, 2019

Uh oh!

Uh oh!

karthikhw commented Oct 6, 2019 •

edited

Loading

virajjasani commented Oct 6, 2019 •

edited

Loading