Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling request_queueing breaks trace ingestion filtration due to dropping http.url tag #3312

Open
tylerwillingham opened this issue Dec 7, 2023 · 2 comments
Assignees
Labels
bug Involves a bug community Was opened by a community member

Comments

@tylerwillingham
Copy link

Current behaviour

When enabling request queuing, the trace root span does not tag the http.url the same way that it does with this option disabled.

For our applications this means that since we opted into the feature our frequent calls to healthcheck endpoints are being ingested again. We currently use the filter_tags option in the APM agent configuration to drop traces. For example:

apm_config:
  # ...
  filter_tags:
    reject:
      - "http.url:/health"
      - "http.url:/internal/metrics"

Expected behaviour

http.proxy.request spans tag http.url

Steps to reproduce

Environment

  • ddtrace version: 1.16.2
  • Configuration block (Datadog.configure ...):
Datadog.configure do |config|
  # ...
  config.tracing.instrument :rails, request_queuing: :exclude_request, web_service_name: "excellent-application"
  # ...
end
  • Ruby version: 3.1.2
  • Operating system: various
  • Relevant library versions:
@tylerwillingham tylerwillingham added bug Involves a bug community Was opened by a community member labels Dec 7, 2023
@TonyCTHsu
Copy link
Contributor

👋 @tylerwillingham , if I understand this correctly, the root span changes from rack.request to http.proxy.request and you are expecting http.proxy.request to contain the http.url tag like rack.request?

@TonyCTHsu TonyCTHsu self-assigned this Jan 11, 2024
@tylerwillingham
Copy link
Author

tylerwillingham commented Jan 11, 2024

@TonyCTHsu yeah exactly - I think that would be the solution for our team

Once we enabled request_queuing our agent-level rejections set via filter_tags stopped working.

Example `filter_tags` agent configuration

      filter_tags:
        reject:
          - "http.url:/health"
          - "http.url:/internal/metrics"

If we opt-out of the request_queuing feature, the traces drop again (I believe because the filter tags are respected once again).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Involves a bug community Was opened by a community member
Projects
None yet
Development

No branches or pull requests

2 participants