Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregate ValueRecorder to HistogramAggregator on Prometheus #1255

Closed
alexppg opened this issue Oct 16, 2020 · 1 comment
Closed

Aggregate ValueRecorder to HistogramAggregator on Prometheus #1255

alexppg opened this issue Oct 16, 2020 · 1 comment
Labels
1.10.0rc1 release candidate 1 for metrics GA bug Something isn't working metrics release:required-for-ga To be resolved before GA release sdk Affects the SDK package.

Comments

@alexppg
Copy link

alexppg commented Oct 16, 2020

Describe your environment
Sorry if I use the wrong nomenclature, I'm beginning to understand how OT works.

I'm trying to use the view example changing the exporter to prometheus.

So far I've seen that:

Traceback (most recent call last):
  File "example_microservice/test.py", line 98, in <module>
    print(generate_latest())
  File "/home/ap/.local/share/virtualenvs/example-microservice-5M-GZT7Z/lib/python3.6/site-packages/prometheus_client/exposition.py", line 137, in generate_latest
    output.append(sample_line(s))
  File "/home/ap/.local/share/virtualenvs/example-microservice-5M-GZT7Z/lib/python3.6/site-packages/prometheus_client/exposition.py", line 103, in sample_line
    line.name, labelstr, floatToGoString(line.value), timestamp)
  File "/home/ap/.local/share/virtualenvs/example-microservice-5M-GZT7Z/lib/python3.6/site-packages/prometheus_client/utils.py", line 9, in floatToGoString
    d = float(d)
TypeError: ("float() argument must be a string or a number, not 'collections.OrderedDict'", Metric(requests_size, size of requests, unknown, , [Sample(name='requests_size', labels={'test': 'value'}, value=OrderedDict([(20, 1), (40, 1), (60, 0), (80, 0), (100, 0), (inf, 1)]), timestamp=None, exemplar=None)]))

Steps to reproduce
Execute this modified version of the view.py example. To see the HistogramAggregator error, just comment the meter.register_view(counter_view2) line.

from opentelemetry import metrics
from opentelemetry.sdk.metrics import (
    MeterProvider,
    Counter,
    ValueRecorder,
)
from opentelemetry.exporter.prometheus import PrometheusMetricsExporter
from opentelemetry.sdk.metrics.export.aggregate import (
    HistogramAggregator,
    LastValueAggregator,
    MinMaxSumCountAggregator,
    SumAggregator,
)
from opentelemetry.sdk.metrics.view import View, ViewConfig
from prometheus_client import generate_latest
from time import sleep

# Use the meter type provided by the SDK package
metrics.set_meter_provider(MeterProvider())
meter = metrics.get_meter(__name__)
metrics.get_meter_provider().start_pipeline(meter, PrometheusMetricsExporter(), 1)

requests_counter = meter.create_metric(
    name="requests",
    description="number of requests",
    unit="1",
    value_type=int,
    metric_type=Counter,
)

requests_size = meter.create_metric(
    name="requests_size",
    description="size of requests",
    unit="1",
    value_type=int,
    metric_type=ValueRecorder,
)

# Views are used to define an aggregation type and label keys to aggregate by
# for a given metric

# Two views with the same metric and aggregation type but different label keys
# With ViewConfig.LABEL_KEYS, all labels but the ones defined in label_keys are
# dropped from the aggregation
counter_view1 = View(
    requests_counter,
    SumAggregator,
    label_keys=["environment"],
    view_config=ViewConfig.LABEL_KEYS,
)
counter_view2 = View(
    requests_counter,
    MinMaxSumCountAggregator,
    label_keys=["os_type"],
    view_config=ViewConfig.LABEL_KEYS,
)
# This view has ViewConfig set to UNGROUPED, meaning all recorded metrics take
# the labels directly without and consideration for label_keys
counter_view3 = View(
    requests_counter,
    LastValueAggregator,
    label_keys=["environment"],  # is not used due to ViewConfig.UNGROUPED
    view_config=ViewConfig.UNGROUPED,
)
# This view uses the HistogramAggregator which accepts an option config
# parameter to specify the bucket ranges
size_view = View(
    requests_size,
    HistogramAggregator,
    label_keys=["environment"],  # is not used due to ViewConfig.UNGROUPED
    aggregator_config={"bounds": [20, 40, 60, 80, 100]},
    view_config=ViewConfig.UNGROUPED,
)

# Register the views to the view manager to use the views. Views MUST be
# registered before recording metrics. Metrics that are recorded without
# views defined for them will use a default for that type of metric
meter.register_view(counter_view1)
meter.register_view(counter_view2)
meter.register_view(counter_view3)
meter.register_view(size_view)

# The views will evaluate the labels passed into the record and aggregate upon
# the unique labels that are generated
# view1 labels will evaluate to {"environment": "staging"}
# view2 labels will evaluate to {"os_type": linux}
# view3 labels will evaluate to {"environment": "staging", "os_type": "linux"}
requests_counter.add(100, {"environment": "staging", "os_type": "linux"})

# Since this is using the HistogramAggregator, the bucket counts will be reflected
# with each record
requests_size.record(25, {"test": "value"})
requests_size.record(-3, {"test": "value"})
requests_size.record(200, {"test": "value"})

sleep(2)

print(generate_latest())
input("...\n")

What is the expected behavior?
It should not return a traceback error and generate the metrics.

What is the actual behavior?
It does return a traceback error and doesn't generate the metrics.

@alexppg
Copy link
Author

alexppg commented Oct 27, 2020

Just to update if anybody finds this issue, @lzchen answered me in the gitter channel. The Histogram aggregation is not implemented yet for prometheus. Also, there's some discussion about which should be the default aggregator for ValueRecord: open-telemetry/opentelemetry-specification#982

@lzchen lzchen added metrics sdk Affects the SDK package. labels Nov 9, 2020
@codeboten codeboten added release:required-for-ga To be resolved before GA release 1.10.0rc1 release candidate 1 for metrics GA labels Nov 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.10.0rc1 release candidate 1 for metrics GA bug Something isn't working metrics release:required-for-ga To be resolved before GA release sdk Affects the SDK package.
Projects
None yet
Development

No branches or pull requests

3 participants