HDDS-1811. Prometheus metrics are broken #1118

adoroszlai · 2019-07-18T06:40:40Z

What changes were proposed in this pull request?

Fix invalid metric type errors:

target=http://192.168.69.76:9882/prom err="invalid metric type \"apache.hadoop.ozone.container.common.transport.server.ratis._csm_metrics_delete_container_avg_time gauge\""

and

target=http://scm:9876/prom err="invalid metric type \"_rati_s-_thre_e-d7116831-ac55-4bf2-a259-d85cfba0572d counter\""

datanode: avoid . in record name by using simple class name
SCM: replace - with _. Also properly convert ALL_CAPS names, eg. RATIS_THREE to ratis_three instead of _rati_s-_thre_e.

https://issues.apache.org/jira/browse/HDDS-1811

How was this patch tested?

Updated unit test.

Checked metrics in ozoneperf pseudo-cluster.

…lid metric

adoroszlai · 2019-07-18T06:41:09Z

/label ozone

hadoop-yetus · 2019-07-18T09:14:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	82	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	27	Maven dependency ordering for branch
+1	mvninstall	533	trunk passed
+1	compile	279	trunk passed
+1	checkstyle	75	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	937	branch has no errors when building and testing our client artifacts.
+1	javadoc	169	trunk passed
0	spotbugs	422	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	636	trunk passed
		_ Patch Compile Tests _
0	mvndep	26	Maven dependency ordering for patch
+1	mvninstall	725	the patch passed
+1	compile	366	the patch passed
+1	javac	366	the patch passed
-0	checkstyle	45	hadoop-hdds: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	897	patch has no errors when building and testing our client artifacts.
+1	javadoc	206	the patch passed
+1	findbugs	677	the patch passed
		_ Other Tests _
-1	unit	423	hadoop-hdds in the patch failed.
-1	unit	2712	hadoop-ozone in the patch failed.
+1	asflicense	57	The patch does not generate ASF License warnings.
		9152

Reason	Tests
Failed junit tests	hadoop.hdds.scm.block.TestBlockManager
	hadoop.hdds.scm.container.placement.algorithms.TestContainerPlacementFactory
	hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
	hadoop.hdds.scm.pipeline.TestNodeFailure
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
	hadoop.ozone.container.server.TestSecureContainerServer
	hadoop.ozone.container.ozoneimpl.TestSecureOzoneContainer
	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
	hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory
	hadoop.ozone.client.rpc.TestReadRetries
	hadoop.ozone.client.rpc.TestBlockOutputStream

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/1/artifact/out/Dockerfile
GITHUB PR	#1118
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 466c89fad20b 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `3dc256e`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/1/artifact/out/diff-checkstyle-hadoop-hdds.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/1/artifact/out/patch-unit-hadoop-hdds.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/1/testReport/
Max. process+thread count	4151 (vs. ulimit of 5500)
modules	C: hadoop-hdds/framework hadoop-hdds/container-service U: hadoop-hdds
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-07-18T12:08:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	135	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	16	Maven dependency ordering for branch
+1	mvninstall	676	trunk passed
+1	compile	334	trunk passed
+1	checkstyle	93	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	1147	branch has no errors when building and testing our client artifacts.
+1	javadoc	204	trunk passed
0	spotbugs	397	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	686	trunk passed
		_ Patch Compile Tests _
0	mvndep	28	Maven dependency ordering for patch
+1	mvninstall	564	the patch passed
+1	compile	298	the patch passed
+1	javac	298	the patch passed
+1	checkstyle	100	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
-1	shadedclient	40	patch has errors when building and testing our client artifacts.
-1	javadoc	35	hadoop-hdds in the patch failed.
-1	javadoc	65	hadoop-ozone in the patch failed.
+1	findbugs	630	the patch passed
		_ Other Tests _
-1	unit	397	hadoop-hdds in the patch failed.
-1	unit	3542	hadoop-ozone in the patch failed.
+1	asflicense	47	The patch does not generate ASF License warnings.
		9227

Reason	Tests
Failed junit tests	hadoop.hdds.scm.container.placement.algorithms.TestContainerPlacementFactory
	hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
	hadoop.ozone.client.rpc.TestContainerStateMachineFailures
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.client.rpc.TestBCSID
	hadoop.ozone.client.rpc.TestCloseContainerHandlingByClient
	hadoop.ozone.om.TestScmSafeMode
	hadoop.ozone.container.server.TestSecureContainerServer
	hadoop.ozone.web.client.TestBuckets
	hadoop.ozone.web.client.TestKeys
	hadoop.ozone.dn.scrubber.TestDataScrubber
	hadoop.ozone.client.rpc.TestWatchForCommit
	hadoop.ozone.container.ozoneimpl.TestSecureOzoneContainer
	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
	hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory
	hadoop.ozone.client.rpc.TestReadRetries
	hadoop.ozone.client.rpc.TestBlockOutputStream

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/artifact/out/Dockerfile
GITHUB PR	#1118
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 1c03a2eeb675 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `23e9beb`
Default Java	1.8.0_212
javadoc	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/artifact/out/patch-javadoc-hadoop-hdds.txt
javadoc	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/artifact/out/patch-javadoc-hadoop-ozone.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/artifact/out/patch-unit-hadoop-hdds.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/testReport/
Max. process+thread count	4536 (vs. ulimit of 5500)
modules	C: hadoop-hdds/framework hadoop-hdds/container-service U: hadoop-hdds
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

adoroszlai · 2019-07-18T15:23:24Z

@lokeshj1703 @mukul1987 please review

hadoop-yetus · 2019-07-19T11:56:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	523	Docker mode activated.
		_ Prechecks _
+1	dupname	1	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	20	Maven dependency ordering for branch
+1	mvninstall	484	trunk passed
+1	compile	245	trunk passed
+1	checkstyle	60	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	796	branch has no errors when building and testing our client artifacts.
+1	javadoc	147	trunk passed
0	spotbugs	317	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	518	trunk passed
		_ Patch Compile Tests _
0	mvndep	18	Maven dependency ordering for patch
+1	mvninstall	426	the patch passed
+1	compile	254	the patch passed
+1	javac	254	the patch passed
+1	checkstyle	66	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	654	patch has no errors when building and testing our client artifacts.
+1	javadoc	154	the patch passed
+1	findbugs	541	the patch passed
		_ Other Tests _
+1	unit	282	hadoop-hdds in the patch passed.
-1	unit	2074	hadoop-ozone in the patch failed.
+1	asflicense	42	The patch does not generate ASF License warnings.
		7461

Reason	Tests
Failed junit tests	hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
	hadoop.ozone.client.rpc.TestCloseContainerHandlingByClient
	hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures
	hadoop.ozone.container.server.TestSecureContainerServer
	hadoop.ozone.container.ozoneimpl.TestSecureOzoneContainer
	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
	hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
	hadoop.ozone.client.rpc.TestFailureHandlingByClient
	hadoop.ozone.client.rpc.TestOzoneAtRestEncryption

Subsystem	Report/Notes
Docker	Client=18.09.8 Server=18.09.8 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/3/artifact/out/Dockerfile
GITHUB PR	#1118
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 2a9658119c44 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `4e66cb9`
Default Java	1.8.0_212
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/3/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/3/testReport/
Max. process+thread count	5008 (vs. ulimit of 5500)
modules	C: hadoop-hdds/framework hadoop-hdds/container-service U: hadoop-hdds
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1118/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

anuengineer

+1, LGTM

anuengineer · 2019-07-22T21:35:25Z

Thank you for your contribution. I have committed this patch to trunk and 0.4.1 branch.

…ache#1118) This reverts commit ec536e1. PR 1116 included some issues, which will be fixed in PR 1117: 1) semantic change to return values of IncomingMessageEnvelope.getEventTime and IncomingMessageEnvelope.getArrivalTime (changed from millis to nanos) which are public API methods, and it will be hard to find existing usages to make sure they are updated properly, 2) usages of constructor of IncomingMessageEnvelope was not updated to handle nanos correctly in certain places (e.g. KafkaConsumerProxy)

adoroszlai added 2 commits July 17, 2019 23:29

HDDS-1811. Prometheus metrics are broken for datanodes due to an inva…

3339095

…lid metric

HDDS-1811. Prometheus metrics are broken for SCM

1ca11fb

elek added the ozone label Jul 18, 2019

HDDS-1811. Fix checkstyle

09583ff

anuengineer approved these changes Jul 22, 2019

View reviewed changes

anuengineer closed this Jul 22, 2019

adoroszlai deleted the HDDS-1811 branch July 23, 2019 06:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-1811. Prometheus metrics are broken #1118

HDDS-1811. Prometheus metrics are broken #1118

Uh oh!

adoroszlai commented Jul 18, 2019

Uh oh!

adoroszlai commented Jul 18, 2019

Uh oh!

hadoop-yetus commented Jul 18, 2019

Uh oh!

hadoop-yetus commented Jul 18, 2019

Uh oh!

adoroszlai commented Jul 18, 2019

Uh oh!

hadoop-yetus commented Jul 19, 2019

Uh oh!

anuengineer left a comment

Uh oh!

anuengineer commented Jul 22, 2019

Uh oh!

Uh oh!

HDDS-1811. Prometheus metrics are broken #1118

HDDS-1811. Prometheus metrics are broken #1118

Uh oh!

Conversation

adoroszlai commented Jul 18, 2019

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

adoroszlai commented Jul 18, 2019

Uh oh!

hadoop-yetus commented Jul 18, 2019

Uh oh!

hadoop-yetus commented Jul 18, 2019

Uh oh!

adoroszlai commented Jul 18, 2019

Uh oh!

hadoop-yetus commented Jul 19, 2019

Uh oh!

anuengineer left a comment

Choose a reason for hiding this comment

Uh oh!

anuengineer commented Jul 22, 2019

Uh oh!

Uh oh!