prometheus exporter precision error with histogram bucket

[cjp421]

Component(s)

exporter/prometheus

Describe the issue you're reporting

Overview

When leveraging the prometheus exporter from the otel-collector-contrib project, the values for the distribution buckets are getting corrupted/improper precision on some of the buckets when viewing the metrics in prometheus.

Setup

• Spring Boot 3.1.6 API instrumented with the otel-javaagent.
• Distribution buckets configured as follows: management.metrics.distribution.slo[http.server.requests]: 10ms,20ms,30ms,40ms,50ms,60ms,70ms,80ms,90ms,100ms,200ms,300ms,400ms,500ms,600ms,700ms,800ms,900ms,1000ms,2000ms,3000ms,4000ms,5000ms,6000ms,7000ms,8000ms,9000ms,10000ms

Using docker.io/otel/opentelemetry-collector-contrib:0.97.0

Relevant otel-config.yaml:

receivers:
  otlp:
   protocols:
      http:
      grpc:
exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
    enable_open_metrics: true

  debug:
    verbosity: detailed

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [debug, prometheus]

When hitting the exposed prometheus metric scrape endpoint, some of the le label values for the http_server_requests_seconds_bucket have unexpected extra precision added to them.

# HELP http_server_requests_seconds 
# TYPE http_server_requests_seconds histogram
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.01"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.02"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.030000000000000002"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.04"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.05"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.060000000000000005"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.07"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.08"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.09000000000000001"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.1"} 0
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.2"} 1
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.30000000000000004"} 2
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.4"} 2
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.5"} 2
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.6000000000000001"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.7000000000000001"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.8"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="0.9"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="1"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="2"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="3"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="4"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="5"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="6"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="7"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="8"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="9"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="10"} 3
http_server_requests_seconds_bucket{error="none",exception="none",job="my-app",method="GET",outcome="SUCCESS",status="200",uri="/services/{id}",le="+Inf"} 3

Expected Results

le="0.030000000000000002" should be le="0.03"
le="0.060000000000000005" should be le="0.06"
le="0.09000000000000001" should be le="0.09"
le="0.30000000000000004" should be le="0.3"
le="0.6000000000000001" should be le="0.6"
le="0.7000000000000001" should be le="0.7"

Otel Collector Debug Output

Relevant output from verbose debug logging (the ExplicitBounds values appear to be correct here):

Metric #38
Descriptor:
     -> Name: http.server.requests
     -> Description: 
     -> Unit: s
     -> DataType: Histogram
     -> AggregationTemporality: Cumulative
HistogramDataPoints #0
Data point attributes:
     -> error: Str(none)
     -> exception: Str(none)
     -> method: Str(GET)
     -> outcome: Str(SUCCESS)
     -> status: Str(200)
     -> uri: Str(/services/{id})
StartTimestamp: 2024-04-18 13:14:49.236021 +0000 UTC
Timestamp: 2024-04-18 13:16:49.243748 +0000 UTC
Count: 3
Sum: 0.978216
Min: 0.196685
Max: 0.573594
ExplicitBounds #0: 0.010000
ExplicitBounds #1: 0.020000
ExplicitBounds #2: 0.030000
ExplicitBounds #3: 0.040000
ExplicitBounds #4: 0.050000
ExplicitBounds #5: 0.060000
ExplicitBounds #6: 0.070000
ExplicitBounds #7: 0.080000
ExplicitBounds #8: 0.090000
ExplicitBounds #9: 0.100000
ExplicitBounds #10: 0.200000
ExplicitBounds #11: 0.300000
ExplicitBounds #12: 0.400000
ExplicitBounds #13: 0.500000
ExplicitBounds #14: 0.600000
ExplicitBounds #15: 0.700000
ExplicitBounds #16: 0.800000
ExplicitBounds #17: 0.900000
ExplicitBounds #18: 1.000000
ExplicitBounds #19: 2.000000
ExplicitBounds #20: 3.000000
ExplicitBounds #21: 4.000000
ExplicitBounds #22: 5.000000
ExplicitBounds #23: 6.000000
ExplicitBounds #24: 7.000000
ExplicitBounds #25: 8.000000
ExplicitBounds #26: 9.000000
ExplicitBounds #27: 10.000000
Buckets #0, Count: 0
Buckets #1, Count: 0
Buckets #2, Count: 0
Buckets #3, Count: 0
Buckets #4, Count: 0
Buckets #5, Count: 0
Buckets #6, Count: 0
Buckets #7, Count: 0
Buckets #8, Count: 0
Buckets #9, Count: 0
Buckets #10, Count: 1
Buckets #11, Count: 1
Buckets #12, Count: 0
Buckets #13, Count: 0
Buckets #14, Count: 1
Buckets #15, Count: 0
Buckets #16, Count: 0
Buckets #17, Count: 0
Buckets #18, Count: 0
Buckets #19, Count: 0
Buckets #20, Count: 0
Buckets #21, Count: 0
Buckets #22, Count: 0
Buckets #23, Count: 0
Buckets #24, Count: 0
Buckets #25, Count: 0
Buckets #26, Count: 0
Buckets #27, Count: 0
Buckets #28, Count: 0

[github-actions]

Pinging code owners:

• exporter/prometheus: @Aneurysm9

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/prometheus: @Aneurysm9

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.