HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs #3523

gjacoby126 · 2021-07-22T20:02:36Z

No description provided.

Apache-HBase · 2021-07-22T21:14:12Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 48s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 17s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 44s	master passed
+1 💚	compile	5m 48s	master passed
+1 💚	checkstyle	2m 13s	master passed
+1 💚	spotbugs	4m 49s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 33s	the patch passed
+1 💚	compile	5m 47s	the patch passed
+1 💚	javac	5m 47s	the patch passed
+1 💚	checkstyle	2m 29s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	21m 8s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	5m 22s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 44s	The patch does not generate ASF License warnings.
		69m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3523
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux c400644e6549 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `d15f3cb`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-common hbase-zookeeper hbase-server hbase-testing-util U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-07-22T23:15:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 8s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 36s	master passed
+1 💚	compile	2m 29s	master passed
+1 💚	shadedjars	8m 43s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 48s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 30s	the patch passed
+1 💚	compile	2m 38s	the patch passed
+1 💚	javac	2m 38s	the patch passed
+1 💚	shadedjars	8m 50s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 44s	the patch passed
		_ Other Tests _
+1 💚	unit	2m 5s	hbase-common in the patch passed.
+1 💚	unit	0m 47s	hbase-zookeeper in the patch passed.
+1 💚	unit	146m 15s	hbase-server in the patch passed.
+1 💚	unit	1m 45s	hbase-testing-util in the patch passed.
		190m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3523
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 3a2d0e0bb829 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `d15f3cb`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/testReport/
Max. process+thread count	3866 (vs. ulimit of 30000)
modules	C: hbase-common hbase-zookeeper hbase-server hbase-testing-util U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-07-23T01:01:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 17s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 30s	master passed
+1 💚	compile	2m 16s	master passed
+1 💚	shadedjars	9m 18s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 31s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 8s	the patch passed
+1 💚	compile	2m 7s	the patch passed
+1 💚	javac	2m 7s	the patch passed
+1 💚	shadedjars	9m 18s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 31s	the patch passed
		_ Other Tests _
+1 💚	unit	2m 2s	hbase-common in the patch passed.
+1 💚	unit	0m 48s	hbase-zookeeper in the patch passed.
+1 💚	unit	252m 31s	hbase-server in the patch passed.
+1 💚	unit	2m 21s	hbase-testing-util in the patch passed.
		297m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3523
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6f3642ee8fe1 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `d15f3cb`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/testReport/
Max. process+thread count	2741 (vs. ulimit of 30000)
modules	C: hbase-common hbase-zookeeper hbase-server hbase-testing-util U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3523/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2021-07-23T01:49:14Z

hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtil.java

@@ -40,7 +40,9 @@
 /**
 * Common helpers for testing HBase that do not depend on specific server/etc. things.
 */
-@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.PHOENIX)
+@InterfaceAudience.LimitedPrivate({HBaseInterfaceAudience.COPROC,


Ah, in general let's not do this.

Mark it as IA.LP("Phoenix") is already a compromise, let's not add more compromise here...

I'm not sure I understand what the compromise is here?

For example, I wear three different hats. I write code for HBase itself, for Phoenix, and for my dayjob. You seem to be arguing that the type of test infrastructure I require depends on what hat I'm wearing, not on the code I'm writing. Are you saying that server-side component tests are good when I wear the HBase hat, a tolerated compromise when I wear the Phoenix hat, and bad when I wear the dayjob hat? And that that would be the case _even for the same code _ depending on where I intended to push it when it's done?

In my view, HBase tests that just exercise the client API should use the TestingHBaseCluster. Same with Phoenix tests, or dayjob tests that only rely on the HBase client API. Tests that exercise server-side components that need the extra control over server-side state should be able to do so. That's true whether the components are housed in an HBase repo, or a Phoenix repo, or any other project, open source or not.

HBase gives a lot of ways to create server side code, and they all need to be tested. The IA should recognize that.

I'm not a native English speaker so sorry I can not understand what you mean.

What I say compromise here, is in HBASE-13126, we want to mark HBTU as IA.Private, but consider the current situation, at least we need to mark it as IA.LP("Phoenix"), as we are sure that Phoenix needs to test something internal to HBase. This is a compromise.

But if you offer your help on improving the new testing clsuter interface, I think maybe we could finish the support for Phoenix before 3.0.0 is out, then probably we could mark HBTU as IA.Private in 3.0.0 finally.

Thanks.

Apache9 · 2021-07-23T01:51:15Z

hbase-testing-util/src/main/java/org/apache/hadoop/hbase/testing/ServerTestingHBaseCluster.java

+   *
+   * @return the utility running the minicluster
+   */
+  HBaseTestingUtil getUtil();


This is a no no.

Let's find another way to only expose the necessary methods for testing coprocessors and other server side hooks such as tags. The point here is, HBaseTestingUtil is a class, not an interface, which makes very hard for the HBase developper to make it stable while keeping it clean.

@Apache9 - thanks, that's a good point. In the next revision of this patch, I will create an interface for HBTU (and probably its superclasses) so this interface doesn't return a concrete class. I'm going to be on vacation for about a week but will take this up when I get back.

gjacoby126 · 2021-08-31T22:45:53Z

Closing for now as HBASE-26157 is an alternate solution to the same problem

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs

d5ef085

gjacoby126 requested review from Apache9, saintstack and apurtell July 22, 2021 20:02

Apache9 requested changes Jul 23, 2021

View reviewed changes

gjacoby126 closed this Aug 31, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs #3523

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs #3523

Uh oh!

gjacoby126 commented Jul 22, 2021

Uh oh!

Apache-HBase commented Jul 22, 2021

Uh oh!

Apache-HBase commented Jul 22, 2021

Uh oh!

Apache-HBase commented Jul 23, 2021

Uh oh!

Apache9 Jul 23, 2021

Uh oh!

gjacoby126 Jul 23, 2021 •

edited

Loading

Uh oh!

Apache9 Jul 23, 2021

Uh oh!

Apache9 Jul 23, 2021

Uh oh!

gjacoby126 Jul 23, 2021

Uh oh!

gjacoby126 commented Aug 31, 2021

Uh oh!

Uh oh!

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs #3523

HBASE-26115 - ServerTestHBaseCluster interface for testing coprocs #3523

Uh oh!

Conversation

gjacoby126 commented Jul 22, 2021

Uh oh!

Apache-HBase commented Jul 22, 2021

Uh oh!

Apache-HBase commented Jul 22, 2021

Uh oh!

Apache-HBase commented Jul 23, 2021

Uh oh!

Apache9 Jul 23, 2021

Choose a reason for hiding this comment

Uh oh!

gjacoby126 Jul 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Apache9 Jul 23, 2021

Choose a reason for hiding this comment

Uh oh!

Apache9 Jul 23, 2021

Choose a reason for hiding this comment

Uh oh!

gjacoby126 Jul 23, 2021

Choose a reason for hiding this comment

Uh oh!

gjacoby126 commented Aug 31, 2021

Uh oh!

Uh oh!

gjacoby126 Jul 23, 2021 •

edited

Loading