HBASE-27890 Expose a getter on Connection/AsyncConnection for getting… #5257

frostruan · 2023-05-27T15:48:35Z

… public access to connection metrics

frostruan · 2023-05-27T15:57:07Z

I have completed a simple version. I believe there must be a lot of improvement. If you have time, would you mind taking a look and giving me some comments and suggestions?

Apache-HBase · 2023-05-27T16:06:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 2s	#5257 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5257
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/1/console
versions	git=2.25.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-27T16:06:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 2s	#5257 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5257
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/1/console
versions	git=2.25.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-27T16:06:14Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#5257 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5257
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/1/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

… public access to connection metrics

Apache-HBase · 2023-05-28T09:17:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 58s	master passed
+1 💚	compile	3m 8s	master passed
+1 💚	checkstyle	0m 48s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	2m 15s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 41s	the patch passed
+1 💚	compile	3m 5s	the patch passed
+1 💚	javac	3m 5s	the patch passed
+1 💚	checkstyle	0m 47s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	10m 13s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 24s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		37m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5257
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 8fc00b2a1d35 5.4.0-1097-aws #105~18.04.1-Ubuntu SMP Mon Feb 13 17:50:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `79c985f`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-28T12:49:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 52s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 47s	master passed
+1 💚	compile	1m 6s	master passed
+1 💚	shadedjars	4m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 44s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 40s	the patch passed
+1 💚	compile	1m 7s	the patch passed
+1 💚	javac	1m 7s	the patch passed
+1 💚	shadedjars	4m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 42s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 27s	hbase-client in the patch passed.
-1 ❌	unit	223m 23s	hbase-server in the patch failed.
		249m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5257
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d6d6efbfc16f 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `79c985f`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/testReport/
Max. process+thread count	4616 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-28T13:11:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 16s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 23s	master passed
+1 💚	compile	0m 58s	master passed
+1 💚	shadedjars	4m 32s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 21s	the patch passed
+1 💚	compile	0m 56s	the patch passed
+1 💚	javac	0m 56s	the patch passed
+1 💚	shadedjars	5m 21s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 53s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 51s	hbase-client in the patch passed.
-1 ❌	unit	245m 38s	hbase-server in the patch failed.
		270m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5257
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c5ba642bab99 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `79c985f`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/testReport/
Max. process+thread count	4206 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5257/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2023-05-30T03:14:02Z

hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnectionSnapshot.java

Better make an interface and let the MetricsConnection implements it, where we only expose read only methods. Like OnlineRegions and MutableOnlineRegions

I'm also wondering if we want to try to avoid exposing the codahale metrics Snapshot directly? It might make compatibility tricky in the future, if the upstream api there changes? Just a thought

Apache9 · 2023-07-05T14:32:46Z

Any updates here?

Thanks.

frostruan · 2023-07-05T16:43:50Z

Sorry, got a little stuck here. I haven't thought of a good way to expose the metric information in a read-only mode without introducing too many new data structures. I will continue to try.

Thanks Duo.

bbeaudreault · 2023-07-05T23:32:37Z

How many do you think we need to introduce?

Apache9 · 2023-07-06T02:17:12Z

I do not think we need to introduce new 'data structures', instead, we need to introduce new 'interfaces', where the current data structures will implement these interfaces, and provide read only access to the metrics. Anyway, maybe we need to add lots of new methods.

Thanks.

frostruan · 2023-07-07T17:56:52Z

Thanks all for the comments. @Apache9 @bbeaudreault

Many of our metrics are Histogram or Timer types. In order to avoid user code updating metrics, we can only provide interfaces similar to snapshots. As we can imagine, our metric interfaces need to define getMin, getMax, getMean, getMedian, get99thPercentile, get999thPercentile .... , as Duo said, a lot of new methods need to be added, and I am still thinking about how to add them. Thanks.

HBASE-27890 Expose a getter on Connection/AsyncConnection for getting…

26a3e9c

… public access to connection metrics

frostruan force-pushed the HBASE-27890 branch from b25b318 to 26a3e9c Compare May 28, 2023 08:28

Apache9 reviewed May 30, 2023

View reviewed changes

NihalJain mentioned this pull request Jun 1, 2023

HBASE-27885 expose metaCacheHits in MetricsConnection #5253

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-27890 Expose a getter on Connection/AsyncConnection for getting… #5257

HBASE-27890 Expose a getter on Connection/AsyncConnection for getting… #5257

frostruan commented May 27, 2023

Uh oh!

frostruan commented May 27, 2023

Uh oh!

Apache-HBase commented May 27, 2023

Uh oh!

Apache-HBase commented May 27, 2023

Uh oh!

Apache-HBase commented May 27, 2023

Uh oh!

Apache-HBase commented May 28, 2023

Uh oh!

Apache-HBase commented May 28, 2023

Uh oh!

Apache-HBase commented May 28, 2023

Uh oh!

Apache9 May 30, 2023

Uh oh!

bbeaudreault Jun 5, 2023

Uh oh!

Apache9 commented Jul 5, 2023

Uh oh!

frostruan commented Jul 5, 2023

Uh oh!

bbeaudreault commented Jul 5, 2023

Uh oh!

Apache9 commented Jul 6, 2023

Uh oh!

frostruan commented Jul 7, 2023

Uh oh!

Uh oh!

HBASE-27890 Expose a getter on Connection/AsyncConnection for getting… #5257

Are you sure you want to change the base?

HBASE-27890 Expose a getter on Connection/AsyncConnection for getting… #5257

Conversation

frostruan commented May 27, 2023

Uh oh!

frostruan commented May 27, 2023

Uh oh!

Apache-HBase commented May 27, 2023

Uh oh!

Apache-HBase commented May 27, 2023

Uh oh!

Apache-HBase commented May 27, 2023

Uh oh!

Apache-HBase commented May 28, 2023

Uh oh!

Apache-HBase commented May 28, 2023

Uh oh!

Apache-HBase commented May 28, 2023

Uh oh!

Apache9 May 30, 2023

Choose a reason for hiding this comment

Uh oh!

bbeaudreault Jun 5, 2023

Choose a reason for hiding this comment

Uh oh!

Apache9 commented Jul 5, 2023

Uh oh!

frostruan commented Jul 5, 2023

Uh oh!

bbeaudreault commented Jul 5, 2023

Uh oh!

Apache9 commented Jul 6, 2023

Uh oh!

frostruan commented Jul 7, 2023

Uh oh!

Uh oh!