Description
Problem - use jaeger_kafka (Kafka OTEL exporter) to export data directly from Jaeger OTEL Agent to kafka?
The default exporter in Jaeger OTEL Agent is jaeger. I tried to override the default exporter to jaeger_kafka(Kafka OTEL exporter). I am wondering if this is possible and here are my attempts to do this (no success yet):
(1) --kafka.producer.topic kafka.producer.brokers: Jaeger OTEL Agent does not recognize those flags (expected).
(2) use OTEL format config file:
spec:
containers:
- args:
- --config=/config/otel-agent-config.yml
---
data:
otel-agent-config: |
exporters:
jaeger_kafka:
topic: jaeger-test
brokers: <some_broker_url>:9092
service:
pipelines:
traces:
receivers: [jaeger]
exporters: [jaeger_kafka]
The logs from jaeger-agent look like:
2020-05-19T00:15:19.474Z INFO builder/exporters_builder.go:281 Exporter is enabled. {"component_kind": "exporter", "exporter": "jaeger_kafka"}
2020-05-19T00:15:19.474Z WARN builder/exporters_builder.go:238 Exportee is not associated with any pipeline and will not export data. {"component_kind": "exporter", "component_type": "jaeger", "component_name": "jaeger"}
github.com/open-telemetry/opentelemetry-collector/service/builder.(*ExportersBuilder).buildExporter
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/builder/exporters_builder.go:238
github.com/open-telemetry/opentelemetry-collector/service/builder.(*ExportersBuilder).Build
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/builder/exporters_builder.go:174
github.com/open-telemetry/opentelemetry-collector/service.(*Application).setupPipelines
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/service.go:301
github.com/open-telemetry/opentelemetry-collector/service.(*Application).setupConfigurationComponents
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/service.go:277
github.com/open-telemetry/opentelemetry-collector/service.(*Application).execute
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/service.go:399
github.com/open-telemetry/opentelemetry-collector/service.New.func1
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/service.go:150
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v0.0.6/command.go:840
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v0.0.6/command.go:945
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v0.0.6/command.go:885
github.com/open-telemetry/opentelemetry-collector/service.(*Application).Start
github.com/open-telemetry/opentelemetry-collector@v0.3.1-0.20200512031848-f588c89b4778/service/service.go:450
main.main
command-line-arguments/main.go:88
runtime.main
runtime/proc.go:203
2020-05-19T00:15:19.474Z INFO service/service.go:306 Starting exporters...
2020-05-19T00:15:19.474Z INFO builder/exporters_builder.go:89 Exporter is starting... {"component_kind": "exporter", "component_type": "jaeger_kafka", "component_name": "jaeger_kafka"}
2020-05-19T00:15:19.474Z INFO builder/exporters_builder.go:94 Exporter started. {"component_kind": "exporter", "component_type": "jaeger_kafka", "component_name": "jaeger_kafka"}
2020-05-19T00:15:19.474Z INFO builder/exporters_builder.go:89 Exporter is starting... {"component_kind": "exporter", "component_type": "jaeger", "component_name": "jaeger"}
2020-05-19T00:15:19.474Z INFO builder/exporters_builder.go:94 Exporter started. {"component_kind": "exporter", "component_type": "jaeger", "component_name": "jaeger"}
2020-05-19T00:15:19.474Z INFO builder/pipelines_builder.go:188 Pipeline is enabled. {"pipeline_name": "traces", "pipeline_datatype": "traces"}
2020-05-19T00:15:19.474Z INFO service/service.go:319 Starting processors...
2020-05-19T00:15:19.474Z INFO builder/pipelines_builder.go:51 Pipeline is starting... {"pipeline_name": "traces", "pipeline_datatype": "traces"}
2020-05-19T00:15:19.474Z INFO builder/pipelines_builder.go:61 Pipeline is started. {"pipeline_name": "traces", "pipeline_datatype": "traces"}
2020-05-19T00:15:19.474Z INFO builder/receivers_builder.go:223 Receiver is enabled. {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger", "datatype": "traces"}
2020-05-19T00:15:19.474Z INFO service/service.go:331 Starting receivers...
2020-05-19T00:15:19.474Z INFO builder/receivers_builder.go:74 Receiver is starting... {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger"}
2020-05-19T00:15:19.474Z INFO builder/receivers_builder.go:79 Receiver started. {"component_kind": "receiver", "component_type": "jaeger", "component_name": "jaeger"}
2020-05-19T00:15:19.474Z INFO healthcheck/handler.go:128 Health Check state change {"component_kind": "extension", "status": "ready"}
2020-05-19T00:15:19.474Z INFO service/service.go:233 Everything is ready. Begin running and processing data.
It seems no error, but I launched a kafka console consumer and did not observe data produced to the topic, when applications generates some spans from upstream to Jaeger OTEL Agent.
As title, I would like to know if Jaeger OTEL Agent can export data to Kafka? If yes, how to do it correctly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment