influxdb2otel receiver #1760

EHSchmitt4395 · 2024-09-25T18:22:21Z

PR Description

Influx receiver that converts to OTEL
Can be used in conjunction with otelcol.exporter.prometheus to further convert influx to prom format.

ex alloy config:

logging {
  level = "debug"
}

// Configure the InfluxDB receiver to collect and forward metrics
otelcol.receiver.influxdb "influxdb_metrics" {
  endpoint = "localhost:8086"  //InfluxDB metrics ingestion endpoint
  output {
    metrics = [otelcol.exporter.prometheus.influx_output.input]  // Forward metrics to Prometheus exporter
  }
}

// Configure the Prometheus exporter to send the metrics to Mimir
otelcol.exporter.prometheus "influx_output" {
  forward_to = [prometheus.remote_write.mimir.receiver]  // Forward metrics to Prometheus remote write (Mimir)
}


prometheus.remote_write "mimir" {
  endpoint {
    url = "https://prometheus-prod-13-prod-us-east-0.grafana.net/api/prom/push"

    basic_auth {
      username = "11111"
      password = "xxxx=="
    }
  }
}

Notes to the Reviewer

plz let me know if issue or something missing with my pr - cheers :)

PR Checklist

CHANGELOG.md updated
Documentation added
Tests updated

clayton-cornell

Docs look good. Only a couple really minor suggestions.

docs/sources/reference/components/otelcol/otelcol.receiver.influxdb.md

…luxdb.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

wildum

thanks for your contribution :) I left a few comments.
The config converter is also missing, have a look at this PR as example:

wildum · 2024-09-27T10:27:12Z

internal/component/otelcol/receiver/influxdb/influxdb.go

+func init() {
+	component.Register(component.Registration{
+		Name:      "otelcol.receiver.influxdb",
+		Stability: featuregate.StabilityExperimental, // Adjust the stability level if needed


upstream stability is "beta" which is pretty stable in Otel standards. I think that it's ok to set it to GenerallyAvailable

wildum · 2024-09-27T10:32:59Z

docs/sources/reference/components/otelcol/otelcol.receiver.influxdb.md

+  endpoint = "localhost:8086"  // InfluxDB metrics ingestion endpoint
+
+  output {
+    metrics = [otelcol.exporter.prometheus.influx_output.input]  // Forward metrics to Prometheus exporter


converting Otel metrics to Prometheus metrics is possible but often unnecessary if the endpoint supports OTLP. Mimir supports OTLP and we also have an OTLP gateway that supports all signals. Here I would just put "metrics = [...]" and in the examples below it's fine to have an example that converts to prometheus but I would put first an example that uses the otlphttp exporter

wildum · 2024-09-27T10:44:42Z

docs/sources/reference/components/otelcol/otelcol.receiver.influxdb.md

+
+| Name                 | Type           | Description                                                      | Default            | Required |
+| -------------------- | -------------- | ---------------------------------------------------------------- | ------------------ | -------- |
+| `endpoint`           | `string`       | `host:port` to listen for traffic on.                            | `"localhost:8086"` | no       |


documentation is missing for the full http server config (args and blocks). See the zipkin receiver config as example

wildum · 2024-09-27T10:45:39Z

internal/component/all/all.go

@@ -94,6 +94,7 @@ import (
 	_ "github.com/grafana/alloy/internal/component/otelcol/processor/tail_sampling"          // Import otelcol.processor.tail_sampling
 	_ "github.com/grafana/alloy/internal/component/otelcol/processor/transform"              // Import otelcol.processor.transform
 	_ "github.com/grafana/alloy/internal/component/otelcol/receiver/datadog"                 // Import otelcol.receiver.datadog
+	_ "github.com/grafana/alloy/internal/component/otelcol/receiver/influxdb"                 // Import otelcol.exporter.datadog


the comment is wrong here

wildum · 2024-09-27T10:46:17Z

docs/sources/reference/components/otelcol/otelcol.receiver.influxdb.md

+title: otelcol.receiver.influxdb
+---
+
+<span class="badge docs-labels__stage docs-labels__item">Experimental</span>


if we put it generally available we should remove the span here

wildum · 2024-09-27T10:52:58Z

go.mod

@@ -813,9 +813,21 @@ require (
 )

 require (
+	github.com/influxdata/influxdb-client-go/v2 v2.14.0
+	github.com/influxdata/influxdb1-client v0.0.0-20220302092344-a9ab5670611c
+	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/influxdbreceiver v0.108.0


would be better to put the requirement next to the other otel components requirements to keep them grouped

wildum · 2024-09-27T10:53:37Z

CHANGELOG.md

@@ -52,7 +52,7 @@ v1.4.0
 - (_Experimental_) Add an `otelcol.processor.interval` component to aggregate metrics and periodically
  forward the latest values to the next component in the pipeline.
 - (_Experimental_) Add a `loki.secretfilter` component to redact secrets from collected logs.
-
+- Added InfluxDB receiver that converts influx metric into OTEL. (@EHSchmitt4395)


this line is misplaced, it should be in the "Main" area above

EHSchmitt4395 added 2 commits September 25, 2024 11:16

initial commit for influxdb2otel receiver

c682925

changelog

48fef74

EHSchmitt4395 added the enhancement New feature or request label Sep 25, 2024

EHSchmitt4395 requested review from clayton-cornell and a team as code owners September 25, 2024 18:22

clayton-cornell reviewed Sep 25, 2024

View reviewed changes

clayton-cornell requested a review from a team September 25, 2024 19:52

EHSchmitt4395 and others added 3 commits September 25, 2024 14:27

Update docs/sources/reference/components/otelcol/otelcol.receiver.inf…

7b84d5a

…luxdb.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update docs/sources/reference/components/otelcol/otelcol.receiver.inf…

b46b285

…luxdb.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

Update docs/sources/reference/components/otelcol/otelcol.receiver.inf…

3ea94e1

…luxdb.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>

wildum requested changes Sep 27, 2024

View reviewed changes

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Oct 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

influxdb2otel receiver #1760

influxdb2otel receiver #1760

EHSchmitt4395 commented Sep 25, 2024

clayton-cornell left a comment

wildum left a comment

wildum Sep 27, 2024

wildum Sep 27, 2024

wildum Sep 27, 2024

wildum Sep 27, 2024

wildum Sep 27, 2024

wildum Sep 27, 2024

wildum Sep 27, 2024

influxdb2otel receiver #1760

Are you sure you want to change the base?

influxdb2otel receiver #1760

Conversation

EHSchmitt4395 commented Sep 25, 2024

PR Description

Notes to the Reviewer

PR Checklist

clayton-cornell left a comment

Choose a reason for hiding this comment

wildum left a comment

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment

wildum Sep 27, 2024

Choose a reason for hiding this comment