Otel exporter not working with Kafka

[harshalschaudhari]

Component(s)

exporter/kafka

What happened?

Description

The application generates traces that can be viewed in Jaeger. I attempted to configure the Otel collector to send these traces to Kafka using various combinations within a Docker Compose setup, but encountered the following error:

Steps to Reproduce

Expected Result

Kafka shows the traces in respective topic.

Actual Result

2024-07-17 16:56:13 2024-07-17T11:26:13.794Z info service@v0.104.0/service.go:115 Setting up own telemetry...
2024-07-17 16:56:13 2024-07-17T11:26:13.799Z info service@v0.104.0/telemetry.go:96 Serving metrics {"address": ":8888", "level": "Normal"}
2024-07-17 16:56:13 2024-07-17T11:26:13.816Z info service@v0.104.0/service.go:193 Starting otelcol... {"Version": "0.104.0", "NumCPU": 2}
2024-07-17 16:56:13 2024-07-17T11:26:13.820Z info extensions/extensions.go:34 Starting extensions...
2024-07-17 16:56:14 2024-07-17T11:26:14.592Z info service@v0.104.0/service.go:256 Starting shutdown...
2024-07-17 16:56:14 2024-07-17T11:26:14.592Z info extensions/extensions.go:59 Stopping extensions...
2024-07-17 16:56:14 2024-07-17T11:26:14.592Z info service@v0.104.0/service.go:270 Shutdown complete.
2024-07-17 16:56:14 Error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp [::1]:9092: connect: connection refused
2024-07-17 16:56:14 2024/07/17 11:26:14 collector server run finished with error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp [::1]:9092: connect: connection refused

Collector version

0.104.0

Environment information

Environment

Docker-Compose

otel-collector:     image: otel/opentelemetry-collector:0.104.0 #otel/opentelemetry-collector:latest     container_name: otel-collector     volumes:       - ./otel-config.yaml:/etc/otel/config.yaml     ports:       - "127.0.0.1:4317:4317"       - "127.0.0.1:55679:55679"     command: ["--config", "/etc/otel/config.yaml"]     networks:       - otel-network     depends_on:       - kafka

OpenTelemetry Collector configuration

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:

exporters:
  logging:
    loglevel: debug
  kafka:
    brokers: ["localhost:9092"]
    topic: "traces"
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, kafka]

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[francois07]

I believe this is related to #24029 TL;DR if Kafka is not available on startup the collector crashes

[harshalschaudhari]

Note: I have tried out kafka using docker with port 29092

Pod Logs as below

2024-07-18 16:25:51 Error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp 172.20.0.9:29092: connect: connection refused
2024-07-18 16:25:51 2024/07/18 10:55:51 collector server run finished with error: cannot start pipelines: kafka: client has run out of available brokers to talk to: dial tcp 172.20.0.9:29092: connect: connection refused

Additional information
I am able to connect Kafka using Offset Explorer 3.0.

OTEL Configuration

exporters:
  kafka:
    brokers:
      - kafka:9092

[hchaudhari-conga]

any update?

[sydneydf]

Hey, Came to make an issue but my issue is near exact same.

Currently using alloy which leverages this component as a wrapper. Exact same error.

Fails with this component when reaching confluent cloud 9092 over a azure privatelinkservice connection (we have kafka connectors in the same subnet operating with no problems.)

Would be really nice to get this working as my organisation has some very unique DMZ requirements for a new project that this would solve perfectly.

[atoulme]

brokers: ["localhost:9092"] will not work on a docker env. localhost means the lo network interface inside the container. Please provide the right host for the kafka endpoint.