Description
openedon Jun 22, 2022
Current behaviour
We have been working on upgrading from Ruby 2.6.5 to Ruby 3.1.0.
With such upgrade, we had to upgrade also dd-trace gem version from 0.54.1 to 1.1.0
Once we have done it successfully, seems we have lost APM traces data for our service.
The traces lost are the incoming received HTTP Requests to our service (rack-request).
Notice in the picture the release time of Ruby 3.1.0 + ddtrace 1.1.0.
It is weird because it seems, we lost some API traces but some others appear (some /v3/first_resource appear, but /v3/another_resource don't appear).
Same for lots of ActiveAdmin traces lost.
Checkout also the number of errors has been lost
so in general, we suspect we are loosing data for HTTP.
Meanwhile, it seems in terms of logs, we have not lost any data (this controller APM has disappeared for instance)
By contrast, for sidekiq for instance, all is working fine.
So, potentially, it might be related with our rack or rails configuration, but we don't see what could be the issue 🤔 .
Expected behaviour
ALL the HTTP requests are traced correctly in the APM.
Before upgrade configuration
Ruby
2.6.5
dd-trace
0.54.1
require './components/platform/base/graphql/schema.rb'
require File.expand_path('../../lib/datadog_tracer', __dir__)
require 'datadog/statsd'
require 'ddtrace'
Datadog.configure do |c|
service = 'web'
hostname = DatadogTracer.resolve_datadog_hostname
c.tracer(
env: Rails.env,
priority_sampling: true,
hostname: hostname,
tags: DatadogTracer.tags,
enabled: ENV['DATADOG_TRACER_ENABLED'].present?)
c.runtime_metrics_enabled = true
c.runtime_metrics(statsd: Datadog::Statsd.new(hostname, DatadogTracer.statsd_port))
c.use(:rack)
c.use(:rails, service_name: service, distributed_tracing: true)
c.use(:sidekiq, analytics_enabled: true, service_name: "#{service}-sidekiq", client_service_name: "#{service}-sidekiq")
c.use(:rake, analytics_enabled: true, service_name: "#{service}-rake", quantize: { args: { show: :all } })
c.use(:aws, service_name: "#{service}-aws")
c.use :redis, service_name: "#{service}-redis-cache"
c.use :redis, describes: { host: ENV['REDIS_HOST'] }, service_name: "#{service}-redis-sidekiq"
c.use(:elasticsearch, service_name: "#{service}-elasticsearch")
c.use(:shoryuken, service_name: "#{service}-shoryuken")
# web backend-graphql-api
c.use :graphql, schemas: [::Platform::Base::GraphQL::Schema], analytics_enabled: true, service_name: ENV['BACKEND_GRAPHQL_API'].present? ? 'backend-graphql-api' : 'web'
end
Current configuration (issue)
Ruby
3.1.0
dd-trace
1.1.0
require './components/platform/base/graphql/schema.rb'
require File.expand_path('../../lib/datadog_tracer', __dir__)
require 'datadog/statsd'
require 'ddtrace'
Datadog.configure do |c|
service = 'web'
hostname = DatadogTracer.resolve_datadog_hostname
c.env = Rails.env
c.agent.host = hostname
c.tags = DatadogTracer.tags
# runtime
c.runtime_metrics.enabled = true
c.runtime_metrics.statsd = Datadog::Statsd.new(hostname, DatadogTracer.statsd_port)
# tracing
c.tracing.enabled = ENV['DATADOG_TRACER_ENABLED'].present?
c.tracing.priority_sampling = true
c.tracing.report_hostname = true
c.tracing.log_injection = true
c.tracing.partial_flush.enabled = true
# instrumenting
c.tracing.instrument(:action_mailer, service_name: "#{service}-action-mailer")
c.tracing.instrument(:active_job, service_name: "#{service}-active-job")
c.tracing.instrument(:active_record, service_name: "#{service}-active-record")
c.tracing.instrument(:active_support, service_name: "#{service}-active-support")
c.tracing.instrument(:aws, service_name: "#{service}-aws")
c.tracing.instrument(:elasticsearch, service_name: "#{service}-elasticsearch")
c.tracing.instrument(:mysql2, service_name: "#{service}-mysql2")
c.tracing.instrument(:rack)
c.tracing.instrument(:rails, service_name: service, distributed_tracing: true)
c.tracing.instrument(:rake, analytics_enabled: true, service_name: "#{service}-rake", quantize: { args: { show: :all } })
c.tracing.instrument(:redis, service_name: "#{service}-redis-cache")
c.tracing.instrument(:redis, describes: { host: ENV['REDIS_HOST'] }, service_name: "#{service}-redis-sidekiq")
c.tracing.instrument(:shoryuken, service_name: "#{service}-shoryuken")
c.tracing.instrument(:sidekiq, analytics_enabled: true, service_name: "#{service}-sidekiq", client_service_name: "#{service}-sidekiq")
# web backend-graphql-api
c.tracing.instrument(:graphql, schemas: [::Platform::Base::GraphQL::Schema], analytics_enabled: true, service_name: ENV['BACKEND_GRAPHQL_API'].present? ? 'backend-graphql-api' : 'web')
end
Current issue Environment
- ddtrace version:
1.1.0 - Ruby version:
3.1.0 - Operating system:
- Relevant library versions:
Rails 6.1.6
