Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New component: RabbitMQ (data) Receiver #34242

Open
3 tasks
frzifus opened this issue Jul 24, 2024 · 2 comments
Open
3 tasks

New component: RabbitMQ (data) Receiver #34242

frzifus opened this issue Jul 24, 2024 · 2 comments
Labels
needs triage New item requiring triage Sponsor Needed New component seeking sponsor

Comments

@frzifus
Copy link
Member

frzifus commented Jul 24, 2024

The purpose and use-cases of the new component

Since version v0.104.0 the opentelemetry collector includes a RabbitMQ exporter. While a rabbitmq receiver exists, it is only used to retrieve node metadata.

With the enhanced ability to read from the queue with an OpenTelemetry Collector, it can be used to reroute, manipulate, or further process traffic. #28891 (comment)

Example configuration for the component

Minimal receiver options required to ensure feature parity with exporter

receivers:
  rabbitmq:
    connection:
      endpoint: amqp://localhost:5672 # (required) The RabbitMQ endpoint to connect to.
      vhost # (optional) The RabbitMQ virtual host to connect to.
      tls: # (optional): TLS configuration.
        insecure: true
        ...
      auth:
        plain: # (optional) Configuration if using SASL PLAIN authentication 
          username: user
          password: pass
    queue:
      name: otlp_spans
    encoding_extension: otlp_encoding/rabbitmq #  (required) ID of the encoding/decoding extension used to unmarshal data.

extensions:
  otlp_encoding/rabbitmq:
    protocol: otlp_json

Telemetry data types supported

Logs, metrics, and traces

Is this a vendor-specific component?

  • This is a vendor-specific component
  • If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
  • If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

@frzifus

Sponsor (optional)

No response

Additional context

Since an exporter and receiver already exist, does it make sense to add the desired functionality to the receiver?

Perhaps someone has an alternative name suggestion which is better then RabbitMQDataReceiver ? 😅


cc @atoulme @swar8080 @djaglowski @cpheps

@frzifus frzifus added Sponsor Needed New component seeking sponsor needs triage New item requiring triage labels Jul 24, 2024
@frzifus
Copy link
Member Author

frzifus commented Jul 31, 2024

Yesterday I got a poc up and running. It uses the existing retry logic of the rabbitmqexporter.

To avoid hard to maintain copy pasta we may want to move this logic into e.g. internal/rabbitmq so that it can be reused by a potential receiver.

Config

receivers:
  rabbitmqdata:
    encoding_extension: otlp_encoding/rabbitmq
    connection:
      vhost: "/"
      endpoint: amqp://localhost:5672
      auth:
        plain:
          username: user
          password: password
      queue:
        - name: otlp_metrics
          consumer: col1
          exclusive: true

exporters:
  debug:

extensions:
  otlp_encoding/rabbitmq:
    protocol: otlp_json

service:
  extensions: [otlp_encoding/rabbitmq]
  pipelines:
    metrics:
      receivers: [rabbitmqdata]
      exporters: [debug]

Output

image

@swar8080
Copy link
Contributor

swar8080 commented Aug 1, 2024

Kind of a mouthful but rabbitmqmessagereciever is another option for the component id to help distinguish it from rabbitmqreceiver

codeboten added a commit that referenced this issue Aug 14, 2024
…#34361)

This PR moves the retry logic from amqp publisher to amqp connection.
Connection, client and other utility structures have been moved from
`exporter/rabbitmqexporter/internal/publisher` to `internal/rabbitmq`.

**Link to tracking Issue:** #34242


----

cc @swar8080 @atoulme

---------

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
f7o pushed a commit to f7o/opentelemetry-collector-contrib that referenced this issue Sep 12, 2024
…open-telemetry#34361)

This PR moves the retry logic from amqp publisher to amqp connection.
Connection, client and other utility structures have been moved from
`exporter/rabbitmqexporter/internal/publisher` to `internal/rabbitmq`.

**Link to tracking Issue:** open-telemetry#34242


----

cc @swar8080 @atoulme

---------

Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>
Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage New item requiring triage Sponsor Needed New component seeking sponsor
Projects
None yet
Development

No branches or pull requests

2 participants