Description
openedon Jun 8, 2020
In 0.36.0 we have a known issue with stale tracer instances being cached incorrectly on integrations, which can result in the instrumentation using the wrong tracer with old settings. The symptoms of this can manifest in a variety of ways, but all of which make it look as though the tracer has ignored settings, or isn't returning active state as expected.
This issue is the result of an intersection of some new changes to the tracer core (where we rebuild tracers after they are changed, following an immutable pattern #996) and old behavior in the integration configuration model (which will cache an instance of the tracer if the integration is explicitly configured with one, as opposed to falling back to the global Datadog.tracer
.)
This is known to affect Rails and GraphQL, and is suspected to affect other integrations as well. For Rails users, you can try using #1064 to see if it resolves your issue (please let us know if it does, so we can know if it works, and what problems it solves.)
A few of these issues can be worked around by using ENV
vars to drive configuration when possible, but not all of these issues have such workarounds.
We're working on a more comprehensive fix to the integrations such that they will not cache tracers, and an overhaul to our CI to detect this issue; we hope to have that ready soon. In the meantime if you have an issue related to this not mitigated by a workaround, we'd recommend using either 0.34.2 or 0.35.2.
Here's a list of issues that are suspected to be related to this bug. At this time, we assume they all have the same approximate cause as described above.
- After version 0.35.0, traces are getting sent even with
Datadog.tracer.enabled = false
#1029 - 0.36.0 stops services with custom names to show up in time spend #1058
- orphaned tracer instance during rails boot causing nil error #1061
- Datadog not using hostname but 127.0.0.1 #1065
- 0.36.0 regression
Datadog.tracer.active_correlation
is not set for Rails requests #1069 - Web transaction spans disappeared after upgrade #1078