Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve conversion of resource attributes to/from prometheus #2381

Merged
merged 10 commits into from
Mar 5, 2022
Next Next commit
update resource attributes used by prometheus, and round-trip other r…
…esource attributes
  • Loading branch information
dashpole committed Feb 24, 2022
commit af0881b5abece40242a9fd17ecd98a05907b2cc2
16 changes: 9 additions & 7 deletions specification/metrics/datamodel.md
Original file line number Diff line number Diff line change
Expand Up @@ -1177,11 +1177,9 @@ The following attributes MUST be associated with scraped metrics as resource att
| OTLP Resource Attribute | Description |
| ----------------------- | ----------- |
| `service.name` | The configured name of the service that the target belongs to |
| `job` | Identical to `service.name` |
| `instance` | The <host>:<port> of the target's URL that was scraped. |
| `host.name` | `instance` The <host> portion of `instance` |
| `port` | `instance` The <port> portion of `instance` |
| `scheme` | `http` or `https` |
| `net.host.name` | The <host> portion of the target's URL that was scraped |
dashpole marked this conversation as resolved.
Show resolved Hide resolved
| `net.host.port` | The <port> portion of the target's URL that was scraped |
dashpole marked this conversation as resolved.
Show resolved Hide resolved
| `http.scheme` | `http` or `https` |

### OTLP Metric points to Prometheus

Expand Down Expand Up @@ -1240,9 +1238,13 @@ OpenTelemetry Metric Attributes MUST be converted to [Prometheus labels](https:/

#### Resource Attributes

In SDK Prometheus (pull) exporters, all resource attributes MUST be dropped, and MUST NOT be attached as labels. The scraper of the endpoint is expected to discover resource attributes of the endpoint it is scraping.
In SDK Prometheus (pull) exporters, resource attributes SHOULD be converted to the [`target_info`](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) metric, or MUST be dropped, and MUST NOT be attached as labels to other metrics. The `target_info` metric MUST info-typed metric whose labels MUST include the resource attributes, and MUST NOT include any other labels. There MUST be at most one `target_info` metric exposed on a Prometheus endpoint.
dashpole marked this conversation as resolved.
Show resolved Hide resolved

In the Collector's Prometheus pull and push (remote-write) exporters, it is possible for metrics from multiple targets to be sent together, so targets must be disambiguated from one another. However, the Prometheus exposition format and [remote-write](https://github.com/Prometheus/Prometheus/blob/main/prompb/remote.proto) formats do not include a notion of resource, and expect metric labels to distinguish scraped targets. By convention, [`job` and `instance`](https://Prometheus.io/docs/concepts/jobs_instances/#jobs-and-instances) labels distinguish targets and are expected to be present on metrics exposed on a Prometheus pull exporter (a ["federated"](https://Prometheus.io/docs/Prometheus/latest/federation/) Prometheus endpoint) or pushed via Prometheus remote-write. In the collector Prometheus exporters, the `job` and `instance` resource attributes MUST be converted to Prometheus metric labels, and other resource attributes SHOULD NOT be converted to metric labels.
In the Collector's Prometheus pull and push (remote-write) exporters, it is possible for metrics from multiple targets to be sent together, so targets must be disambiguated from one another. However, the Prometheus exposition format and [remote-write](https://github.com/Prometheus/Prometheus/blob/main/prompb/remote.proto) formats do not include a notion of resource, and expect metric labels to distinguish scraped targets. By convention, [`job` and `instance`](https://Prometheus.io/docs/concepts/jobs_instances/#jobs-and-instances) labels distinguish targets and are expected to be present on metrics exposed on a Prometheus pull exporter (a ["federated"](https://Prometheus.io/docs/Prometheus/latest/federation/) Prometheus endpoint) or pushed via Prometheus remote-write. In the collector Prometheus exporters, the `service.name` attribute MUST be converted to the `job` metric label, and the `net.host.name` and `net.host.port` attributes, if present, MUST be converted to the `instance` label as `<net.host.name>:<net.host.port>`. If `job` and `instance` are successfully added, other resource attributes SHOULD be converted to a [`target_info`](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems) metric, or MUST be dropped. The `target_info` metric is an info-typed metric whose labels MUST include the resource attributes, and MUST NOT include any other labels other than `job` and `instance`. There MUST be at most one `target_info` metric exported for each unique combination of `job` and `instance`.
bogdandrutu marked this conversation as resolved.
Show resolved Hide resolved

If info-typed metrics are not yet supported by the language Prometheus client library, a gauge-typed metric named `target_info` with a constant value of 1 MUST be used instead.

To convert OTLP resource attributes to Prometheus labels, string Attribute values are converted directly to labels, and non-string Attribute values MUST be converted to string attributes following the [attribute specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/common.md#attribute).

## Footnotes

Expand Down