1.64.0

Note

When the sampling.priority tag has a positive value we now translate that to forceKeep to ensure the trace segment is kept regardless of any previous sampling decision

Components

Application Security Management (IAST)

• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨⚡ Improve performance of regexps in IAST and query obfuscator (#11649 - @manuel-alvarez-alvarez)
• 🐛 Fix IAST JDBC instrumentation when getMetadata throws non-SQLException (#11420 - @smola)

Application Security Management (WAF)

• 🐛 Fix unbounded multipart form-field buffering in Jetty 8 PartHelper (#11876 - @jandro996)
• ✨ Add request_excluded tag to appsec.waf.requests metric (#11744 - @jandro996)
• 🐛 Fix rasp.rule.match metric always reporting block:N/A (#11723 - @jandro996)
• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨ Fix waf.updates metric success:N/A tag from sequential counter metric values (#11604 - @jandro996)
• ✨ Implement SCA Reachability runtime detection: report vulnerable classes and callsites via telemetry (#11352 - @jandro996)
• ✨ Add server.request.body.filenames AppSec address for Vert.x 3.4, 4.0 and 5.0 (#11268 - @jandro996)
• ✨ Add server.request.body.filenames support for Jetty (#10988 - @jandro996)

Build & Tooling

• ✨🔐 Forbid OkHostnameVerifier to prevent accidental use, CVE-2021-0341 (#11694 - @bric3)

Configuration at Runtime

• ✨ Avoid logging stack trace during expected remote-config exception (#11328 - @sarahchen6)

Configuration

• ✨ Add OTLP Trace Metrics Configs (#11653 - @mhlidd)
• ✨⚡ Optimize collection creation and memory allocation of supported config (#11616 - @PerfectSlayer)

Continuous Integration Visibility

• ✨ Avoid final field modifications in Karate instrumentation (#11753 - @daniel-mohedano)
• ✨ Avoid final field modification in Weaver instrumentation (#11728 - @daniel-mohedano)
• 🐛 Disable invokedynamic bytecode for legacy Gradle instrumentation (#11618 - @daniel-mohedano)
• ✨ Support Maven generateTestClasspath lookup on 3.6.0+ (#11577 - @daniel-mohedano)
• ✨ Truncate CI Visibility meta tag values (#11485 - @juan-fernandez)

Crash Tracking

• 💡 Include JVM distribution details in crash report (#11715 - @gyuheon0h)
• ✨ Improve crash report message when signal is absent (#11681 - @amarziali)
• 🐛 Avoid crashtracking temp-script path hijack (#11613 - @jbachorik)

Data Streams Monitoring

• ✨ Update EventBridge DSM topic naming (#11603 - @jeastham1993)

Dynamic Instrumentation

• 🐛 Fix Kotlin Debug source mapping (#11803 - @jpbempel)
• ✨ Add evaluation timeout for expressions (#11741 - @jpbempel)
• 🐛 Fix Spring MethodParameters detection (#11647 - @jpbempel)
• 🐛 Fix sampling log probe with capture expressions (#11571 - @jpbempel)

Metrics

• ✨🧹 Update AggregateEntry to Split OK/Error Latencies (#11719 - @mhlidd)
• ✨⚡ Lazy-allocate error latency histogram on AggregateEntry (#11478 - @dougqh)
• ✨⚡ Introduce per-component / tag cardinality limits in client-side stats (#11387 - @dougqh)

OpenFeature

• 🐛 Use agent OTel metrics for OpenFeature evaluations (#11745 - @sameerank)
• 🐛 Skip malformed FFE flags during config parsing (#11534 - @leoromanovsky)
• 🧹 Use canonical FFE fixtures (#11355 - @leoromanovsky)

Profiling

• 🐛 Bump ddprof to 1.46.0 (#11858 - @jbachorik)
  • Add alternative TLS implementation to fix a crash on musl/aarch64 by @zhengyu123 in DataDog/java-profiler#600
  • Don't preload jmethodIDs when cstack=vm with hotspot JVM by @zhengyu123 in DataDog/java-profiler#549
  • Scope OTEL ThreadContext storage to the carrier thread (PROF-15271) by @rkennke in DataDog/java-profiler#625
  • Close two TOCTOU races between SIGPROF handler and JFR lifecycle by @r1viollet in DataDog/java-profiler#614
• 🐛 Request carrier-scoped ddprof context storage on JDK 21+ (#11826 - @rkennke)
• 🐛 Restore app context attributes wiped by ddprof setContext on span activation (#11646 - @jbachorik)
• 🐛 Publish thread context attribute keys at process start (#11558 - @ivoanjo)
• ✨ Add force jmethodID experimental feature support (#11549 - @zhengyu123)

Telemetry

• ✨ Migrate SCA Reachability to method-level symbol database (sca-reachability-symbols) (#11614 - @jandro996)
• ✨ Fix waf.updates metric success:N/A tag from sequential counter metric values (#11604 - @jandro996)
• ✨ Implement SCA Reachability runtime detection: report vulnerable classes and callsites via telemetry (#11352 - @jandro996)

Trace context propagation

• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨ Implement an Org Propagation Guard mechanism (#11289 - @amarziali)

Tracer core

• ✨ Introduce api-management inferred proxy spans to Azure (#11804 - @TophrC-dd)
• 🧹 Migrate simple try-with-resources use of AgentScope to ContextScope (#11763 - @mcculls)
• ⚡🧹 Overhaul set benchmarks: split Immutable / SingleThreaded, add Set.copyOf (#11721 - @dougqh)
• ✨ Wire -Pjmh.includes and -PtestJvm into internal-api JMH config (#11703 - @dougqh)
• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨ Update jnr-unixsocket dependency to 0.38.25 (#11615 - @mcculls)
• ✨ Update dd-instrument-java dependency to v0.0.4 (#11567 - @mcculls)
• ✨ Update byte-buddy dependency to 1.18.10 (#11565 - @mcculls)
• ✨⚡ Pre-construct TagMap.Entry objects in InternalTagsAdder (#11555 - @dougqh)
• 🐛 Fix JDBC URL parsing crash when no port specified (#11550 - @ValentinZakharov)
• 🐛 Build DatadogClassLoader resource URL from agent jar URL, not OS path (#11480 - @dougqh)
• ✨ Include advice class name in instrumentation exception handler log (#11457 - @amarziali)
• ✨⚡ Use TagMap.set in CoreTracer (#11106 - @dougqh)
• 📖⚡ List iteration benchmark (#10888 - @dougqh)
• 📖⚡ Atomics benchmark (#10862 - @dougqh)
• ✨⚡ Adding lighter String processing methods to Strings (#10640 - @dougqh)

Tracer public API

• 💡⚠️ Translate a sampling.priority tag set to a positive value to forceKeep (#11621 - @mcculls)
• 🐛 Fix DDTraceId/DD64bTraceId class-initialization deadlock (#11509 - @bm1549)

Instrumentations

Apache HttpComponents

• 🐛🔐 Fix private headers leaking to a different origin on redirect (#11671 - @vandonr)

Apache Spark instrumentation

• ✨ Inject ol_env into ol spark run tags (#11786 - @ajyu)

Armeria Instrumentation

• 🐛 Fix Ameria 1.40 instrumentation compatibility (#11827 - @vandonr)
• ✨ Ignore capturing connection continuation for armeria (#11657 - @amarziali)

AWS Lambda instrumentation

• 🐛 Don't add empty headers on lambda completion (#11551 - @amarziali)

AWS SDK instrumentation

• 🐛 Fix message handler spans appear disconnected from the incoming SQS trace (#11511 - @ygree)

Azure Functions instrumentation

• ✨ Introduces the api-management inferred proxy spans to Azure (#11804 - @TophrC-dd)

Core Java language instrumentation

• ✨ Avoid leaking continuations on jdk-http-client (#11557 - @amarziali)

Eclipse Vert.x instrumentation

• ✨ Add support for Vert.x 5.1 in tracing (#11655 - @vandonr)

JAX-RS instrumentation

• ✨ Avoid leaking continuations on internal rxjava schedulers (#11540 - @amarziali)

JDBC instrumentation

• 🐛🔐 Fix password leak from JDBC URL (#11568 - @ValentinZakharov)
• 🐛 Fix JDBC URL parsing crash when no port specified (#11550 - @ValentinZakharov)
• 🐛 Fix IAST JDBC instrumentation when getMetadata throws non-SQLException (#11420 - @smola)

Kafka instrumentation

• 🐛 Add safe extraction for malformed base64 kafka headers (#11472 - @amarziali)

Lettuce instrumentation

• 🐛 Fix Redis command span decoration to resolve node connection for master-replica providers in Lettuce 5+ (#11819 - @ygree)
• 🐛 Fine tune async propagation for lettuce 5 (#11607 - @amarziali)

Netty instrumentation

• ✨ Do not capture continuation on netty GlobalEventExecutor static init (#11654 - @amarziali)
• 🐛 Fix continuation leakage on netty http2 client pipeline (#11595 - @amarziali)

Reactor instrumentation

• 🧹 Make reactor-core / reactive-streams context propagation Context-first (#11578 - @amarziali)

Servlet instrumentation

• 🐛 Fix StackOverflowError in caused by missing re-entrancy guard (#11838 - @ValentinZakharov)
• 🐛 Tolerate servlet 2.x wrappers used on servlet-3 classpath for isAsyncStarted calls (#11542 - @amarziali)

Spring instrumentation

• ✨ Drop the duplicate handler resolution in Spring Web 6 instrumentation (#11533 - @amarziali)
• 🐛 Build DatadogClassLoader resource URL from agent jar URL, not OS path (#11480 - @dougqh)

All other instrumentations

• ✨ Add RxJava 3 instrumentation (#11849 - @ValentinZakharov)
• ✨ Support proper async propagation for GAX retrying futures (#11629 - @amarziali)

1.63.2

Note

This release contains the same changes as in release 1.63.1. Release 1.63.1 is not flawed in any way.

Components

Application Security Management (IAST)

• ✨⚡ Improve performance of regexps in IAST and query obfuscator (#11710 - @manuel-alvarez-alvarez)

Continuous Integration Visibility

• ✨ Support Maven generateTestClasspath lookup on 3.6.0+ (#11634 - @daniel-mohedano)

Dynamic Instrumentation

• 🐛 Fix sampling log probe with capture expressions (#11628 - @jpbempel)

OpenFeature

• 🐛 Skip malformed FFE flags during config parsing (#11648 - @PerfectSlayer)

1.63.1

Note

This release is missing in Docker registries.

Components

Application Security Management (IAST)

• ✨⚡ Improve performance of regexps in IAST and query obfuscator (#11710 - @manuel-alvarez-alvarez)

Continuous Integration Visibility

• ✨ Support Maven generateTestClasspath lookup on 3.6.0+ (#11634 - @daniel-mohedano)

Dynamic Instrumentation

• 🐛 Fix sampling log probe with capture expressions (#11628 - @jpbempel)

OpenFeature

• 🐛 Skip malformed FFE flags during config parsing (#11648 - @PerfectSlayer)

1.63.0

Components

AI Guard

• ✨ Copy anomaly detection tags to AI Guard spans (#11319 - @smola)
• ✨ Collect client IP tags for AI Guard requests (#11233 - @smola)

Application Security Management (WAF)

• ✨ Collect Datadog security-testing headers on HTTP server entry spans (#11418 - @christophe-papazian)
• ✨ Add server.request.body.filenames and files_content for GlassFish/Payara (#11267 - @jandro996)

Configuration at Runtime

• 🐛 Request feature flag config on the first RC poll (#11465 - @leoromanovsky)

Continuous Integration Visibility

• ✨ Propagate Jenkins custom parent ID (#11348 - @juan-fernandez)
• ✨ Update JUnit4 instrumentation to fire test suite events for Bazel (#11322 - @daniel-mohedano)
• ✨ Propagate test skipping enabled tag to CI Visibility test spans (#11300 - @anmarchenko)
• ✨ Use test_srcdir as virtual repo root for bazel runs (#11273 - @daniel-mohedano)
• 🐛 Consider Gradle wrapper as a wrapper for headless calculations (#11266 - @daniel-mohedano)
• ✨ Report bazel as provider in telemetry for payload-in-files mode (#11207 - @daniel-mohedano)

Crash Tracking

• ✨ Redacted registry memory mapping in crash tracking (#11025 - @bric3)

Database Monitoring

• ✨ Add a new dynamic_service DBM propagation mode (#11432 - @amarziali)
• ✨ Improve DBM commenter (#11339 - @amarziali)

Dynamic Instrumentation

• ✨ debugger/symdb: add upload metadata fields to upload event message (#11329 - @andreimatei)

ML Observability (LLMObs)

• 🐛 emit session_id as top-level field + propagate via context (#11371 - @jessicagamio)

Metrics

• ✨⚡ Update client-side stats to use light weight Hashtable (#11382 - @dougqh)
• ✨⚡ Defer MetricKey construction to the aggregator thread (#11381 - @dougqh)
• ✨⚡ Use bitmask SpanKindFilter for per-span eligibility in metrics aggregator (#11380 - @dougqh)
• ✨ Support OTLP runtime metrics with OTel-native naming (#11318 - @link04)
• ✨ Allow ignoring the agent version when checking for trace stats computation support (#11064 - @bric3)

OpenFeature

• 🐛 Recover OpenFeature provider after initialization timeout (#11474 - @leoromanovsky)
• 🐛 Request feature flag config on the first RC poll (#11465 - @leoromanovsky)
• 🐛 use delta temporality for OTLP metrics export (#11426 - @sameerank)
• 🐛 read OTEL_SERVICE_NAME for metrics resource (#11424 - @sameerank)

Profiling

• 🐛 Bump java-profiler to 1.44.0 for more critical stability fixes (#11501 - @jbachorik)
  • 🐛 Bump java-profiler to 1.43.0 for critical stability fixes (#11475 - @jbachorik)
• ✨ Enable publishing process context by default (#11288 - @ivoanjo)

Telemetry

• ✨ Add Telemetry for Extension Finder SPIs (#11298 - @mhlidd)
• ✨ Introduce OTel SPI Metrics (#11297 - @mhlidd)
• ✨ Update Baggage Metrics for Extract (#11278 - @mhlidd)

Testing

• ✨ 🪞 9979 - Record detailed instrumentation errors during tests (#10300 - @deejgregor - thanks for the contribution!)

Trace context propagation

• ✨ Add lazy-transformer to KeyClassifier for deferred header decoding (#11466 - @amarziali)

Tracer core

• ✨⚡ Cache hashCode on UTF8BytesString (#11444 - @dougqh)
• 🐛 Fix DDSpan.addThrowable crashing when exception getMessage() throws (#11428 - @ValentinZakharov)
• 🐛 Avoid extracting the baggage from null values (#11396 - @amarziali)
• ✨⚡ Update client-side stats to use light weight Hashtable (#11382 - @dougqh)
• ✨⚡ Defer MetricKey construction to the aggregator thread (#11381 - @dougqh)
• ✨⚡ Use bitmask SpanKindFilter for per-span eligibility in metrics aggregator (#11380 - @dougqh)
• 🐛 Fix bugs in GenerationalUtf8Cache constructor and access-time handling (#11323 - @dougqh)
• ✨ Remove deprecated internal spanBuilder methods (#11287 - @amarziali)
• 🐛 Fix peer IP tag in Play 2.3/2.4/2.5 (#11274 - @smola)
• ✨ Move network.client.ip out of AppSec into HttpServerDecorator (#11208 - @smola)

Instrumentations

AWS Lambda instrumentation

• ✨ force placeholder resource so the Lambda Extension drops dd-tracer-serverless-span (#11313 - @zarirhamza)

AWS SDK instrumentation

• 🐛⚡ Skip ClassCastException on every DynamoDB request in AWS SDK v2 instrumentation (#11285 - @dougqh)

Eclipse Vert.x instrumentation

• 🐛 finish vertx.route-handler via RoutingContext.addEndHandler fallback (#11312 - @zarirhamza)

gRPC instrumentation

• ✨⚡ Avoid Metadata.key allocations on grpc (#11351 - @amarziali)

JDBC instrumentation

• ✨ Mute telemetry when Connection.getClientInfo fails (#11412 - @amarziali)

JMS instrumentation

• ✨ Have a fully JMS 1.0 compatible instrumentation (#11413 - @amarziali)

OpenTelemetry instrumentation

• ✨⚡ Use FAST_LANE prioritization for OTLP trace export (#11447 - @mcculls)
• ✨ Add Additional OTel JVM Runtime Metrics and Gate "Developmental" Metrics (#11411 - @mhlidd)
• 🐛 Fix the behaviour of asynchronous counters and up-down counters (#11332 - @mcculls)
• 🐛 Avoid potential shutdown delay when using GRPC protocol for OTLP (#11321 - @mcculls)
• ✨ Support OTLP runtime metrics with OTel-native naming (#11318 - @link04)
• 🐛 Fix OtelLogRecordProcessor schedule (#11315 - @mcculls)
• 💡⚡ Reduce allocation rate when marshaling OTLP data (#11296 - @mcculls)
• ✨ Enable publishing process context by default (#11288 - @ivoanjo)

Play Framework instrumentation

• 🐛 Fix peer IP tag in Play 2.3/2.4/2.5 (#11274 - @smola)

Reactor instrumentation

• ✨⚡ avoid attaching context if it's already attached (#11393 - @amarziali)

Servlet instrumentation

• 🐛 Tolerate AbstractMethodError for servletRequest remote port (#11403 - @amarziali)
• 🐛 avoid NPE when headerNames is null (#11394 - @amarziali)

All other instrumentations

• 🧹 Remove deprecated startSpan and use integration name for telemetry (#11181 - @amarziali)

1.62.0

Warning

This version contains a known regression in profiler that can cause crashes in the profiled application. Please upgrade to the latest version.

Note

Do not enable Trace Payload protocol V1 until further notice as it is not enabled on all DataDog backends yet.

Components

AI Guard

• ✨ Add ai_guard.event tag to local root span (#11109 - @manuel-alvarez-alvarez)
• ✨ Expose tag probabilities in SDK responses (#11063 - @manuel-alvarez-alvarez)

Application Security Management (IAST)

• 🐛 🪞 10671 - Fix platform-dependent String.getBytes() calls to use explicit UTF-8 charset (#11149 - @mcculls - thanks @saravadeo for the contribution!)

Application Security Management (WAF)

• ✨ Add server.request.body.files_content AppSec address: commons-fileupload (#11137), Tomcat & Netty multipart (#11198), Jersey & RESTEasy(#11229) - @jandro996
• ✨ Add server.request.body.filenames AppSec address: Jersey & RESTEasy (#11171), Akka HTTP (#11173), Undertow & Play (#11174) - @jandro996
• ✨ Expose server.io.fs.file_write AppSec address for write file operations (#11084),FileOutputStream/FileInputStream constructors(#11113), Files.copy overloads (#11179) - @jandro996
• ✨ Add AppSec Java support for AWS Lambdas (#10570 - @claponcet)
• 🐛 Restore original tracer during LambdaAppSecHandlerTest cleanup (#11172 - @claponcet)

Configuration

• ✨ Add Integration Specific Handling for Config Inversion Linter (#11074 - @mhlidd)
• ✨ Add Profiling Specific Handling for Config Inversion Linter (#11066 - @mhlidd)
• ✨ Update Spock Tests to run with STRICT_TEST mode by default (#11032 - @mhlidd)

Continuous Integration Visibility

• ✨ Honor attempt to fix failures (#11244 - @daniel-mohedano)
• ✨ Avoid feature discover and java.nio.* access in Bazel Mode (#11232 - @daniel-mohedano)
• ✨ Implement Bazel support for CI Visibility (#11150 - @daniel-mohedano)
• 🐛 Fix TestNG session status reported with EFD (#11087 - @daniel-mohedano)

Crash Tracking

• ✨ Enable sending crashtracking reports to errors intake by default (#11243 - @amarziali)
• 🐛 🪞 10671 - Fix platform-dependent String.getBytes() calls to use explicit UTF-8 charset (#11149 - @mcculls - thanks @saravadeo for the contribution!)
• ✨ Do not use java.nio in crashtracking init (premain) (#11080 - @amarziali)

Dynamic Instrumentation

• 🐛 Fix NullPointerException on samplers (#11230 - @jpbempel)
• 🐛 Fix NullPointerException in Exception Replay (#11219 - @jpbempel)
• ✨ Add support for case insensitivity file names (#11205 - @jpbempel)
• 🐛 Fix DebuggerAgent commonInit (#11183 - @jpbempel)
• 🐛 🪞 10671 - Fix platform-dependent String.getBytes() calls to use explicit UTF-8 charset (#11149 - @mcculls - thanks @saravadeo for the contribution!)
• ✨ Add support for windows-style path (backslashes) (#11073 - @jpbempel)

Metrics

• ✨ Report dropped stats aggregates in health metrics (#11231 - @amarziali)
• ✨ Upgrade jmxfetch dependency to 0.52.0 (#11127 - @mcculls)
• ✨ Upgrade java-dogstatsd-client dependency to 4.4.5 (#11126 - @mcculls)

OpenFeature

• 💡 Add flag evaluation metrics via OTel counter and OpenFeature Hook (#11040 - @typotter)
• ✨ Refine variant reason computation to include SPLIT and STATIC (#11037 - @typotter)
• 🐛 Strictly check variant types in Flag Evaluations (#11036 - @typotter)

Profiling

• ✨ Bump Java profiler library to 1.42.0 (#11222 - @jbachorik)
  • (subsumes) Bump Java profiler library to 1.41.0 (#11169 - @jbachorik)

Telemetry

• ✨ Remove noise at boostrap due to telemetry failures (#11218 - @amarziali)

Testing

• 🧹 Migrate OtlpWriterCombinedTest from groovy to junit (#11245 - @mtoffl01)
• ✨ Improve trace assert API (#11209 - @PerfectSlayer)
• 🐛⚡ Improve instrumentation JUnit testing (#11096 - @PerfectSlayer)
• ✨ Improve Config JUnit extension (#11094 - @PerfectSlayer)
• ✨ Add Config and Context JUnit extensions (#11076 - @PerfectSlayer)
• ✨ Update Spock Tests to run with STRICT_TEST mode by default (#11032 - @mhlidd)

Trace context propagation

• 🐛🔐 Add items and bytes limits to baggage extraction (#11265 - @PerfectSlayer)
• 💡 Add Sofa RPC instrumentation (Bolt, H2C, REST, Triple) (#11135 - @ValentinZakharov)
• 🐛 Preload scope classes to prevent virtual thread deadlock (#11111 - @PerfectSlayer)

Tracer core

• 🐛 Fix peer IP tags in Jetty/Dropwizard (#11237 - @smola)
• ✨ Use ParentBasedAlwaysOnSampler by default when OTLP traces export is enabled (#11225 - @mtoffl01)
• ✨ Introduce a generic JavaModuleOpener for JPMS (#11220 - @amarziali)
• ✨ Allow mapping process tags from env or sysprops (#11204 - @amarziali)
• ✨ Add OtlpWriter for OTLP traces export (#11200 - @mtoffl01)
• 🐛 Add support double quotes as a single token in SQLNormalizer (#11197 - @Eldolfin)
• 🐛 Fix NPE in PayloadDispatcherImpl (#11196 - @amarziali)
• 🐛 Widen info state hashing catch (#11192 - @amarziali)
• ✨ Add classloader exclusions for drools generated classes (#11191 - @amarziali)
• 🐛 🪞 10671 - Fix platform-dependent String.getBytes() calls to use explicit UTF-8 charset (#11149 - @mcculls - thanks @saravadeo for the contribution!)
• 🐛 Abort on IBMJ9's keytool (#11133 - @bric3)
• ✨ Upgrade byte-buddy dependency to 1.18.8 (#11128 - @mcculls)
• ✨⚡ Cache span.kind as byte ordinal for fast isOutbound() (#11116 - @dougqh)
• 🐛 Fix container ID parsing for cgroup v1 in EKS-Fargate (#11110 - @vandonr)
• ✨ Optimize PendingTrace span registration and time tracking (#11078 - @bm1549)
• 🐛 Fix _dd.p.ksr formatting to use 6 decimal places (#11068 - @bm1549)
• ✨ Efficient Trace Payload Protocol (v1 protocol) (#10801 - @AlexeyKuznetsov-DD)

Instrumentations

Apache Spark instrumentation

• ✨ Add spark.openlineage.appName tag to spark.application spans (#11246 - @aboitreaud)
• 🐛 Fall back to jobGroupId when jobRunId equals taskRunId (#11199 - @aboitreaud)
• ✨ Instrument QueryExecution.assertAnalyzed() to catch DataFrame analysis failures (#11033 - @aboitreaud)

AWS SDK instrumentation

• 🐛 Fix SNS v2 batch injection bug (#11190 - @ygree)

Core Java language instrumentation

• 🐛 Preload scope classes to prevent virtual thread deadlock (#11111 - @PerfectSlayer)
• ✨ Add a jpms opener for HostNameResolver cache (#11095 - @amarziali)

Jetty instrumentation

• 🐛 Fix peer IP tags in Jetty/Dropwizard (#11237 - @smola)

Kafka instrumentation

• ✨ Add Kafka cluster ID and offset tags to producer/consumer spans (#11107 - @piochelepiotr)

OpenTelemetry instrumentation

• 🐛 Correctly export overflow histogram bucket over OTLP (#11261 - @mcculls)
• 💡 Support sending OpenTelemetry logs over OTLP (http/protobuf) (#11253 - @mcculls)
• 🐛⚡ Miscellaneous OpenTelemetry Metrics fixes (#11240 - @mcculls)
• 💡 Implement OpenTelemetry Logs API (#11224 - @mcculls)
• 💡 Add telemetry attributes to OTLP payloads (#11125 - @mcculls)
• 💡 Provide optimized writers for OpenTelemetry's "trace.proto" wire protocol (#11120 - @mcculls)

Sofa RPC

• 💡 Add Sofa RPC instrumentation (Bolt, H2C, REST, Triple) (#11135 - @ValentinZakharov)

1.61.1

Instrumentations

Core Java language instrumentation

• 🐛 Preload scope classes to prevent virtual thread deadlock (#11131 - @PerfectSlayer)

1.61.0

Warning

This version contains a known regression that can cause application hangs when virtual threads are used heavily at startup, on JDK versions from 21+ up to January 2025. If you use virtual threads, please upgrade to the latest version.

Components

Application Security Management (IAST)

• 🐛 Skip XSS check for freemarker built-in escaping expressions in 2.3.24 instrumentation (#10865 - @jandro996)

Application Security Management (WAF)

• ✨ Add server.request.body.filenames support for Tomcat and Netty 4.1 (#10973 - @jandro996)
• ✨ Add server.request.body.filenames AppSec address for commons-fileupload (#10949 - @jandro996)
• ✨🧪 Return AI Guard SDS findings into SDK Response (#10821 - @obordeau)
• 🐛 Fix ObjectIntrospection exposing JDK internal toString() to the WAF (#10820 - @jandro996)
• ✨ Honor in-app blocking settings by default in AI Guard evaluate (#10818 - @smola)
• 🐛 Fix HttpEndpointPostProcessor incorrectly overwriting span resource name (#10754 - @jandro996)
• ✨ Add http.status_code, error, and http.useragent to inferred proxy spans (#10749 - @jandro996)
• 🐛 Fix content-type extraction for akka-http (#10742 - @jandro996)
• ✨🧪 Attach SDS findings to ai_guard spans (#10721 - @obordeau)
• 🐛 Fix missing http.response.headers.content-type span tag on blocking responses (#10711 - @jandro996)
• 🐛 Fix AWS API Gateway endpoints correlation HTTP span tags - Inferred Proxy Spans (#10561 - @jandro996)

Configuration

• ✨ Guarantee OTel Configs to run through ConfigHelper (#11057 - @mhlidd)
• ✨ Update supported-configurations.json w/ Missing Configs (#11046 - @mhlidd)
• ✨ Update JUnit Tests to run with STRICT_TEST mode by default (#11045 - @mhlidd)
• ✨ Update ConfigHelper to throw error in STRICT_TEST mode and capture undocumented Configs (#11031 - @mhlidd)

Continuous Integration Visibility

• 🐛 Include all candidate paths for duplicate keys in coverage reports (#10997 - @daniel-mohedano)
• ✨ Implement configuration error tagging for all backend requests (#10963 - @daniel-mohedano)
• 🐛 Fix Test Management telemetry (#10962 - @daniel-mohedano)
• ✨ Add support for Weaver 0.12 release (#10903 - @daniel-mohedano)
• ✨ Consider empty parameters as null in TestIdentifier creation (#10902 - @daniel-mohedano)
• ✨ Add error tag for library configuration request (#10856 - @daniel-mohedano)
• ✨ Add Java 26 support to CI Visibility (#10839 - @daniel-mohedano)
• ✨⚡ Use HashingUtils hash in CI Visibility (#10784 - @dougqh)

Crash Tracking

• ✨ Feature flag crashtracking extended info, and improve runtime args filtering (#11048 - @bric3)
• 🐛 Unset ld_preload and dyld_insert_libraries for crashtracker child JVM (#11041 - @gyuheon0h)
• ✨ Report JVM argument flags for crash tracking (#11013 - @bric3)
• ✨ Reports loaded libraries in crashtracking reports (#11000 - @bric3)
• 🐛✨ Add crashtracking addresses and error thread name (#10984 - @bric3)
• ✨ Properly handle OpenJ9 crash report (#10976 - @bric3)
• 🐛 Fix crashtracking siginfo missed some information (#10907 - @bric3)
• 🐛 Change error tracking crashtracker source (#10848 - @amarziali)
• 🐛 Unset inherited JVM env vars in crashtracking scripts (#10819 - @jbachorik)

Database Monitoring

• 🐛 Fix Oracle DBM trace correlation (#10829 - @azhou-datadog)

Dynamic Instrumentation

• 🐛 Fix NullPointerException in ExceptionProbe (#11051 - @jpbempel)
• 🐛 Fix NullPointerException in Fingerprinter (#11049 - @jpbempel)
• 🐛 Fix capture limit for capture expressions (#11042 - @jpbempel)
• 🐛 Fix CodeOrigin for interface endpoints (#11017 - @jpbempel)
• ✨ Optimize Lambda proxy class matching on filtering (#10905 - @jpbempel)
• 🐛 Fix Kotlin fake source mapping (#10880 - @jpbempel)
• 🐛 Fix NullPointerException in SymbolExtractor (#10858 - @jpbempel)
• 🐛 Handle exception from retrieving git info (#10853 - @jpbempel)
• 🐛 Change MethodParameters detection reporting (#10849 - @jpbempel)
• 🐛 Prevent retransforming record with type annotation (#10824 - @jpbempel)
• 🐛 Prevent instrumenting debugger agent classes (#10803 - @jpbempel)
• 🐛 Catch exception from detecting MethodParameters (#10779 - @jpbempel)
• 🐛 Prevent instrumenting first line of a constructor (#10772 - @jpbempel)
• 🐛 Catch exceptions in registerSourceFile (#10752 - @jpbempel)
• 🐛 Use SpringVersion to detect MethodParameters usage (#10750 - @jpbempel)
• 🐛 Fix Spring detection (#10740 - @jpbempel)
• 🐛 Add an error sampler for Log probes (#10638 - @jpbempel)

Logs Product

• 🐛 Fix check raw file injection smoke test (#10920 - @bm1549)

ML Observability (LLMObs)

• ✨ Propagate global dd_tags to LLMObs span tags (#10978 - @ygree)
• 🐛 Fix openai-java payload mapping for responses, tool metadata, and prompt tracking (#10644 - @ygree)

Metrics

• ✨ Add GitCommitSha to the client stats (#10833 - @amarziali)
• 🐛 Report gRPC status code in client-computed stats (#10805 - @bric3)
• ✨⚡ Reduce memory allocation from client-side stats (#10705 - @dougqh)
• ✨ Reduce verbosity of StatsD connection errors (#10618 - @PerfectSlayer)

OpenFeature

• 🐛 Allow null targeting key for static and rule-only flags (#10990 - @leoromanovsky)

Platform

• ✨ Add HTTP component API module (#10770 - @PerfectSlayer)

Profiling

• ✨ Enable live heap profiling by default on safe JVM versions (#11039 - @jbachorik)
• ✨ Update Java profiler library to 1.40.1 (#10995 - @jbachorik)
• ✨ Remove experimental jvmti wallclock profiler (#10872 - @zhengyu123)

Telemetry

• ✨ Update Environment Variable Normalization to not add DD prefix for OTel Configs (#11044 - @mhlidd)
• 🐛 Report dd-java-agent itself as a Maven dependency in SCA telemetry (#10975 - @jandro996)
• ✨ Add stable session ID headers to telemetry requests (#10914 - @khanayan123)
• ✨ Update Telemetry Key Normalization to Use Environment Variables (#10823 - @mhlidd)
• ✨⚡ Replace Objects.hash with HashingUtils.hash in telemetry (#10813 - @dougqh)
• ✨⚡ Reduce allocation overhead of ExceptionLogger / LogCollector (#10684 - @dougqh)

Testing

• ✨ Update JUnit Tests to run with STRICT_TEST mode by default (#11045 - @mhlidd)
• 🐛 Fix final_status workaround (#10852 - @cbeauchesne)
• 🧹 Refactor tests following Java migration (#10733 - @jpbempel)

Trace context propagation

• 🐛 Improve VirtualThread context tracking instrumentation (#11009 - @PerfectSlayer)
• 🐛 Fix VirtualThread support on multiple unmount/remount cycles (#10931 - @PerfectSlayer)

Tracer core

• ✨ Use TagMap.set in CoreTracer tracer tags (#11004 - @dougqh)
• ✨⚡ Replace interceptors ConcurrentSkipListSet with priority ordered copy on write "list" (#11002 - @dougqh)
• ✨⚡ Cache Tracer Host Entry to reduce allocation in RemoteHostnameAdder (#10968 - @dougqh)
• ✨ Add ObjectInputStream.readObject to forbidden apis (#10952 - @dougqh)
• ✨ Add ParentBasedAlwaysOnSampler as default sampler for OTLP trace export mode (#10915 - @mtoffl01)
• 🐛 Handle the case the info response does not contain endpoints / empty (#10838 - @amarziali)
• ✨⚡ Use forEach to avoid Iterator allocation (#10830 - @dougqh)
• ✨⚡ Avoid ArrayList copying from TraceInterceptors (#10828 - @dougqh)
• ✨⚡ Avoid creation of empty CopyOnWriteArrayList for span links (#10822 - @dougqh)
• ✨ Update BaseHash when process tags are updated (#10809 - @amarziali)
• ✨ Add _dd.p.ksr propagated tag for Knuth sampling rate (#10802 - @bm1549)
• ✨ Add capped sampling rate increases (#10715 - @raphaelgavache)
• ✨⚡ Avoid boxing by using TagMap.set in PayloadTagsProcessor (#10964 - @dougqh)
• ✨🧪 Add tracing support for native method calls via Java FFM API (#10718 - @amarziali)

Tracer internal logging

• ✨ Add app log collection config value in toString (#10725 - @jpbempel)

Instrumentations

Apache Spark instrumentation

• 🐛 Fix EMR Spark support by using Seq instead of ArrayBuffer for externalAccums (#10987 - @aboitreaud)
• 🐛 Fix spark application spans status on sql analysis failure (#10981 - @aboitreaud)
• 🐛 Fix sql plan nodes order causing test flakiness (#10911 - @aboitreaud)
• 🐛 Fix flaky test assertion by normalizing column ref and plan_id num (#10757 - @aboitreaud)
• ✨ Capture time app-related metrics for launchers (#10708 - @aboitreaud)
• 🐛 Remove Wrong Spark Spans for Inactive Databricks Clusters (#10651 - @larakulkarni1)
• ✨ Store accumulator-stage lookups directly (#10645 - @charlesmyu)
• ✨ Track external accumulators in tracer instead of using SparkInfo values (#10553 - @charlesmyu)

Armeria Instrumentation

• 🐛 Report gRPC status code in client-computed stats (#10805 - @bric3)

AWS SDK instrumentation

• 🐛 Inject APM trace headers into SQS _datadog message attribute on send (#10843 - @ygree)

Core Java language instrumentation

• 🐛 Improve VirtualThread context tracking instrumentation (#11009 - @PerfectSlayer)
• 🐛 Fix VirtualThread support on multiple unmount/remount cycles (#10931 - @PerfectSlayer)
• 🧹 Disable outdated RMI instrumentation (#10897 - @PerfectSlayer)
• 🐛🔐 Use primitive types for RMI context payload serialization (#10935 - @PerfectSlayer)
• ✨ Mute tracing for a...

1.60.4

Important

This release fixes a possibility of profiler deadlocking a thread if a library intercepting SIGSEGV is present

Components

Profiling

• 🐛 Update Java profiler library to 1.40.1 (#11012 - @jbachorik)

1.60.3

Announcement

Important

This release fixes CVE-2026-33728.

Components

Continuous Integration Visibility

• ✨ Add Java 26 support to CI Visibility (#10929 - @daniel-mohedano)

Profiling

• ✨ Update Java profiler library to 1.40.0 (#10932 - @jbachorik)
  • Java 26 compatibility fix
  • Improve stackwalking performance

Instrumentations

RMI

• 🐛🔐 Use primitive types for RMI context payload serialization (#10937 - @PerfectSlayer)

1.60.2

Components

Application Security Management (WAF)

• 🐛 Add http.response.headers.content-type span tag on blocking responses (#10884 - @jandro996)
• 🐛 Fix ObjectIntrospection exposing JDK internal toString() to the WAF (#10882 - @jandro996)

Profiling

• ✨ Update Java profiler library to 1.39.0 (#10869 - @jbachorik) (contains also 1.38.0)