1.67.1

Announcement

Important

Fixes a security vulnerability, as described in GHSA-9x8v-5f43-c55r.

Components

Tracer core

• ✨ Update default query string obfuscation pattern (#12784 - @jandro996)

Instrumentations

Core Java language instrumentation

• 🐛 Fix startup deadlock with virtual threads (#12788 - @amarziali)

1.67.0

Warning

This version contains a regression that can cause application hangs when virtual threads are used heavily at startup, on JDK versions from 21+ up to January 2025.

Components

AI Guard

• ✨ Apply sensitive data redaction from evaluate responses (#12394 - @manuel-alvarez-alvarez)

Application Security Management (WAF)

• 🐛 Finish HTTP client spans and close scopes when AppSec blocks requests (#12611 - @amarziali)
• 🐛 Complete AWS Lambda AppSec telemetry integration (#12587 - @claponcet)
• 🐛 Restore the no-arg constructors the Moshi reflective adapters require (#12566 - @claponcet)
• 🐛 Fix concurrent access to AppSec derivatives during tracing (#12523 - @claponcet)
• ✨ Add block_failure telemetry to jersey, resteasy, commons-fileupload and okhttp (#12519 - @jandro996)
• ✨ Infer status 200 for implicit-success Lambda responses (#12494 - @claponcet)
• 🐛 Consult AppSec trace rate limiter only when the WAF requests trace retention (#12482 - @claponcet)
• 🐛 Report v1 SDK custom events as sdk_version:v1 in telemetry (#12438 - @claponcet)
• ✨ Parse Lambda AppSec request bodies according to their content type (#12363 - @claponcet)
• ✨ Add block-outcome telemetry for Netty blocking enforcement failures (#12316 - @jandro996)
• 🐛 Fix AppSec instrumentation hooks lost after Remote Config reload (#12271 - @jandro996)
• 🐛 Fix multipart body-map cap bypass and charset handling in RESTEasy (#12085 - @jandro996)

Build & Tooling

• ✨ Provide a way to claim classes from other sources as helpers (#12694 - @mcculls)
• ✨ Add JUnit support for @Flaky conditions (#12620 - @sarahchen6)
• ✨ Allow instrumentations to retransform already-loaded classes (#12610 - @mcculls)
• ✨ Reconfigure Muzzle to use advice scanner (#12551 - @sarahchen6)
• ✨ Discover instrumentation helper classes using advice scanner (#12549 - @sarahchen6)
• ✨ Improve Byte Buddy advice transformation diagnostics (#12439 - @amarziali)
• ✨ Add team freeze guard workflow (#12437 - @cbeauchesne)
• ✨ Implement generic advice scanner (#12370 - @sarahchen6)

Configuration

• 🐛 Remove OTel resource attribute tag limit (#12617 - @MilanGarnier)

Continuous Integration Visibility

• 🐛 Forward JUnit 5 report entries to the original listener (#12697 - @daniel-mohedano)
• 🐛 Report Gradle test tasks with no executed tests as skipped (#12656 - @juan-fernandez)
• 🐛 Report explicitly skipped Maven test runs as skipped (#12640 - @juan-fernandez)
• 🐛 Fix ATR eligibility with fail-fast test ordering (#12508 - @juan-fernandez)
• 🐛 Support latest Robolectric 4.17 (#12478 - @AlexeyKuznetsov-DD)
• ✨ Implement dynamic auto test retries (#12467 - @gnufede)
• 🐛 Fix TIA missing line coverage response field (#12384 - @daniel-mohedano)
• 🐛 Fix crash when class bytecode is unavailable for test coverage (#12380 - @dougqh)

Crash Tracking

• 🐛 Fix crash-tracking script/directory permission lockdown (#12330 - @jbachorik)

Database Monitoring

• ✨⚡ Reduce DBM comment buffer allocation (#12582 - @amarziali)
• 🐛 Fix NPE in JDBC instrumentation when connection has no database type (#12371 - @dougqh)
• ✨ Propagate Oracle dynamic service hash without SQL comments (#12234 - @joelmarcotte)

Dynamic Instrumentation

• 🐛 Reinitialize to the initial config when APM_TRACING is absent (#12660 - @jpbempel)
• 🐛 Fix LinkageError for CodeOrigin (#12545 - @jpbempel)
• 🐛 Fix race in ConfigurationUpdater (#12496 - @jpbempel)
• ✨ Add coordinated sampling for snapshot probes (#12452 - @jpbempel)
• 🐛 Fix NullPointerException in Exception Replay (#12429 - @jpbempel)
• 🐛 Avoid Throwable::toString in ExceptionReplay (#12424 - @jpbempel)

GraalVM native-image

• 🐛 Fix stale _dd.base_service on GraalVM/Mandrel native-image builds (#12571 - @Yeison2020)

ML Observability (LLMObs)

• 🐛 Carry the agent manifest inside metadata so it reaches the backend (#12512 - @ncybul)
• ✨ Emit gen_ai.* attributes on APM spans (#12450 - @mz1119)
• ✨ Propagate LLM Observability context across service boundaries (#12416 - @ncybul)

OpenFeature

• ✨ Send the split serial id on feature flag exposure events (#12432 - @danyal002)
• ✨ Publish dd-openfeature in PR snapshots (#12400 - @leoromanovsky)
• 🐛 Constrain direct EVP intake origin to prevent API key leakage (#12299 - @leoromanovsky)

Profiling

• ✨ Bump ddprof to 1.51.0 (#12648 - @zhengyu123)
• ✨ Avoid Netty queue-timing metadata reads before JFR is ready (#12558 - @amarziali)
• ✨ Avoid queue-timing metadata reads before JFR is ready (#12550 - @amarziali)
• ✨ Expose OTel thread/process context without requiring profiling to be enabled (#12546 - @jandro996)
• 🐛 Create profiler snapshot and JFR CLI temp files under configured tempdir (#12484 - @anandreshmi46)

Telemetry

• ✨ Introduce Tags/OTLP Resource Attributes for OTLP Adoption Metrics (#12516 - @mhlidd)

Trace context propagation

• 🐛 Honour request header tags mapped onto headers (#12695 - @ValentinZakharov)
• 🐛 Fix baggage header escaping of non-ASCII values (#12564 - @RoseannaM)
• 🐛 Prevent OkHttp connection pools from retaining request contexts (#12492 - @amarziali)
• 🐛 Prevent OkHttp TaskRunner from retaining request contexts (#12468 - @amarziali)
• 🧹 Parse OpenTelemetry tracestate member (#12405 - @MilanGarnier)
• ✨ Support OpenTelemetry consistent tracestate sampling (#12397 - @MilanGarnier)

Tracer core

• 🐛 Fix application service names beneath inferred proxy spans (#12553 - @mhlidd)
• 🐛 Fix agent sampling rates being ignored when sampling rules are configured (#12490 - @mhlidd)
• ✨⚡ Use ObjectStore to track context bound to arbitrary objects (#12463 - @mcculls)
• 🧹 Improve concurrent hash table insertion performance (#12462 - @dougqh)
• 🐛 Fix null trace config returned from extracted spans (#12413 - @mcculls)
• 🐛 Keep JDK Unix socket timeouts from stopping trace delivery (#12322 - @bric3)

Instrumentations

Akka and Pekko instrumentation

• 🐛 Release Akka message continuations when discarded to dead letters (#12616 - @amarziali)
• 🐛 Prevent Akka HTTP timeout cleanup from retaining request context (#12615 - @amarziali)
• 🐛 Use context swap for Akka and Pekko HTTP flows (#12507 - @amarziali)
• 🐛 Prevent Pekko callbacks from retaining request contexts (#12675 - @amarziali)
• 🐛 Prevent Pekko stream lifecycle context retention (#12470 - @amarziali)

Apache Spark instrumentation

• ✨ Add Spark physical plan only to completed spans (#12645 - @aboitreaud)
• 🐛 Preserve Spark application failures after Spark stops on YARN (#12596 - @aboitreaud)

AWS Lambda instrumentation

• 🐛 Complete AWS Lambda AppSec telemetry integration (#12587 - @claponcet)

AWS SDK instrumentation

• 🐛 Tag http.status_code on AWS SDK spans for non-2xx service errors (#12749 - @Yeison2020)
• ✨⚡ Reduce SQS body propagation parsing overhead (#12568 - @amarziali)
• ✨⚡ Reduce SQS propagation attribute formatting overhead (#12562 - @amarziali)

Core Java language instrumentation

• ✨ Preserve trace context in delayed ForkJoinPool tasks (#12600 - @amarziali)
• 🐛 Fix Java HTTP async continuation lifecycle (#12498 - @amarziali)
• 🐛 Fix continuation cleanup when TimerTask is cancelled (#12464 - @amarziali)
• ✨ Reduce virtual-thread context-propagation overhead on park/unpark (#11893 - @amarziali)

Couchbase instrumentation

• 🐛 Prevent Couchbase Netty initialization from capturing context (#12520 - @amarziali)

Eclipse Vert.x instrumentation

• 🐛 Prevent Vert.x 4 session reaper from retaining request contexts (#12667 - @bric3)

Guidewire instrumentation

• ✨ Enable Guidewire instrumentation by default (#12594 - @ValentinZakharov)

JAX-RS instrumentation

• 🐛 Fix multipart body-map cap bypass and charset handling in RESTEasy (#12085 - @jandro996)
• ✨ Add block_failure telemetry to jersey, resteasy, commons-fileupload and okhttp (#12519 - @jandro996)

JDBC instrumentation

• ✨ Propagate Oracle dynamic service hash without SQL comments (#12234 - @joelmarcotte)

Jetty instrumentation

• 🐛 Preserve Jetty 12 request task invocation types (#12586 - @amarziali)

Kotlin instrumentation

• 🐛 Prevent duplicate Kotlin coroutine continuation release (#12466 - @amarziali)

Lettuce instrumentation

• 🐛 Fix Redis peer tags for Lettuce cluster-mode connections and commands (#12433 - @ygree)

MongoDB instrumentation

• ✨⚡ Avoid copying redacted MongoDB binary values (#12581 - @amarziali)

Netty instrumentation

• ✨⚡ Reuse Netty request-context attributes between requests (#12584 - @amarziali)
• 🧹 Fix context propagation for delayed Netty scheduled task (#12541 - @amarziali)
• 🐛 Prevent Netty idle timers from retaining request context (#12503 - @amarziali)
• 🐛 Prevent Netty periodic purge tasks from inheriting context (#12499 - @amarziali)
• ✨ Add block-outcome telemetry for Netty blocking enforcement failures (#12316 - @jandro996)

Spring instrumentation

• ✨ Share the WebFlux parsed-route cache across requests (#12561 - @amarziali)
• ✨⚡ Avoid repeated controller attribute-key allocation in Spring MVC (#12547 - @amarziali)

Square OkHttp instrumentation

• ✨⚡ Reduce OkHttp AppSec header conversion overhead (#12567 - @amarziali)
• ✨ Add block_failure...

1.66.0

Components

Application Security Management (IAST)

• ✨ Add IAST code injection detection for BeanShell (#12113 - @claponcet)

Application Security Management (WAF)

• ✨ Add API Security schema extraction and missing-route metrics (#12300 - @jandro996)
• 🐛 Bound unresolved-artifact retry loop in ScaReachabilityTransformer (#12245 - @jandro996)
• ✨ Add HTTP span tags to the AWS Lambda invocation span (#12226 - @claponcet)
• ✨ Bump libsqreen (libddwaf-java) to 17.5.0 for libddwaf 2.0.1 (#12118 - @jandro996)

Configuration

• 🐛 Support OpenTelemetry deployment environment names (#12324 - @tylfin)

Continuous Integration Visibility

• 🐛 Avoid empty headless test sessions (#12146 - @daniel-mohedano)

Crash Tracking

• 🐛 Add is_crash tag to crash report (#12156 - @gyuheon0h)

Data Streams Monitoring

• ✨ Capture Kafka consumer group membership on join (#11989 - @piochelepiotr)

Dynamic Instrumentation

• 🐛 Disable Exception Replay for JDK < 11 (#12411 - @jpbempel)
• ✨🧹 Add telemetry metrics for dropped/skipped events (#12319 - @jpbempel)
• 🧹 Add additional collection classes as unsafe (#12280 - @jpbempel)
• 🐛 Make snapshot list for ThrowableState concurrent (#12270 - @jpbempel)
• 🐛 Fix redaction marker for log template (#12269 - @jpbempel)
• 🐛 Fix memory leak with FastThrow optimization (#12254 - @jpbempel)
• 🐛 Add null checks for calls to getAllLoadedClasses (#12236 - @jpbempel)
• 🐛 Fix error spam from filtered out log probes (#12207 - @tylfin)
• 🐛 Close jar entry streams during Symbol Database scan (#12143 - @tylfin)

ML Observability (LLMObs)

• 🐛 Fix OpenAI Java 4.54 compatibility (#12340 - @AlexeyKuznetsov-DD)
• ✨ Add annotateAgentManifest manual API to LLMObs SDK (#12318 - @yahya-mouman)
• ✨ Add agent_version tag with subtree propagation to LLM Observability spans (#12291 - @ncybul)
• ✨ Add head-based sampling for LLM Observability traces (#12277 - @ncybul)
• ✨ Add in LLM Observability Prompt Tracking API (#12161 - @sabrenner)
• ✨ Add Support for Annotating Embedding and Retrieval Documents (#12149 - @sabrenner)
• 🐛 Aggregate span.finished telemetry counts per interval (#12136 - @Yun-Kim)
• ✨ Add LLMObs.submitFeedback for end-user feedback submission (#12129 - @ddog-thibault-nadin)
• ✨ Add in LLM Observability Span Processor API (#12112 - @sabrenner)

Metrics

• 🐛 Fix OTLP trace-metrics attribute compliance gaps (#12144 - @mabdinur)
• ✨ Report stats span collapses over telemetry (client-side stats) (#12070 - @dougqh)

OpenFeature

• 🐛✨ Support arbitrary semver parts (#12286 - @greghuels)
• 🐛 Fix span enrichment singleton initialization (#12273 - @sameerank)
• ✨ Add direct flagevaluation fallback (#12204 - @leoromanovsky)
• ✨ Send agentless Feature Flags exposures directly to EVP (#12195 - @leoromanovsky)
• ✨ Add semantic version operators for feature flag evaluation (#12182 - @greghuels)
• ✨ Emit server-side EVP flagevaluation (#11639 - @leoromanovsky)
• 🐛 Handle gracefully malformed flag evaluations; migrate Date->Instant (#11535 - @leoromanovsky)

Profiling

• 🐛 Bump ddprof to 1.50.0 (#12337 - @jbachorik)
• 🧹 Revert carrier-scoped ddprof context storage export (#11826) (#12302 - @rkennke)
• ✨ Add support for native_socket ddprof config (#12175 - @kaahos)
• 🐛 Bump java-profiler (ddprof) to 1.49.0 (#12168 - @jbachorik)

Telemetry

• ✨ Add API Security schema extraction and missing-route metrics (#12300 - @jandro996)
• 🐛 Aggregate span.finished telemetry counts per interval (#12136 - @Yun-Kim)
• 🐛 Guard core-metric telemetry drain against queue-full delta loss (#12081 - @dougqh)
• ✨ Redact OTLP header and Datadog key configs in telemetry (#11583 - @bm1549)

Trace context propagation

• 🐛 Fix sfn x-datadog-tags input (#12320 - @ojproductions)
• 🐛 Adding baggage limits for OT Baggage (#12303 - @mhlidd)

Tracer core

• 🐛 Preserve deterministic sampler rates (#12382 - @MilanGarnier)
• ✨ Support using GlobalObjectStore for non-field-injected contexts (#12381 - @mcculls)
• 🐛 Fix span link span ID hex padding (#12279 - @itsamenathan - thanks for the contribution!)
• 🐛 Only warn about trace buffer overflow when a kept trace is lost (#12348 - @dougqh)
• ✨ Update dd-instrument-java to v0.0.5 (#12344 - @mcculls)
• ✨ Update byte-buddy dependency to 1.18.12 (#12325 - @sarahchen6)
• 💡 Allow tracer clock to be resynced after an AWS Lambda SnapStart restore (#11882 - @sheldon-nadarajah-sft - thanks for the contribution!)
• ✨ Add agent attribution to Java SDK (#12238 - @yahya-mouman)
• 🐛 Fix uncaught StackOverflowError in StackTraces.getStackTrace (#12233 - @dougqh)
• 🐛 Fix trailing element separator rejecting valid dd tracestate (#12229 - @mcculls)
• ⚡🧹 Recalibrate the UTF8 caches every N spans instead of per span (#11945 - @dougqh)
• 🧹 Wire mergedTracerTags as a read-through parent at span build (level-split) (phase 1a) (#11932 - @dougqh)

Tracer internal logging

• 🐛 Only warn about trace buffer overflow when a kept trace is lost (#12348 - @dougqh)

Instrumentations

Apache Spark instrumentation

• 🐛 Route OpenLineage over the Agent's Unix Domain Socket URL (#12408 - @pawel-big-lebowski)

AWS SDK instrumentation

• 🐛 Fix sfn x-datadog-tags input (#12320 - @ojproductions)
• 🐛 Fix SQS queue name tags from QueueUrl (#12159 - @ygree)

Core Java language instrumentation

• 🐛 Prevent NPE in HttpClientDecorator.isAgentRequest for a null request (#12376 - @dougqh)
• 🐛 Fix NullPointerException in OSGi BundleWiringHelper when bundle reference is stale (#12241 - @dougqh)

Guidewire instrumentation

• 💡 Propagate trace context across Guidewire WSI SOAP worker threads (#12125 - @ValentinZakharov)

Kafka instrumentation

• ✨ Capture Kafka consumer group membership on join (#11989 - @piochelepiotr)

Netty instrumentation

• 🐛 Treat Netty native client aborts as non-error responses (#12295 - @ygree)

OpenTelemetry instrumentation

• 🐛 Fix concurrent span event recording in OTel shim (#12053 - @Meemaw - thanks for the contribution!)

1.65.1

Components

OpenFeature

• 🐛 Handle gracefully malformed flag evaluations; migrate Date->Instant (#12176 - @leoromanovsky)

Profiling

• 🐛 Update java-profiler (ddprof) to 1.49.0 (#12173 - @kaahos)

  Most significant changes/fixes in ddprof 1.49.0:

  • Missing libgcc_s.so.1 on the host no longer causes a crash — DataDog/java-profiler#689, DataDog/java-profiler#720 (Alpine/musl)
  • Avoid retaining invalid VM method pointers — DataDog/java-profiler#699
  • Guard Profiler::recordSample() stack walk against faults — DataDog/java-profiler#687
  • Guard Recording::writeElement against SIGSEGV on a corrupted metadata tree — DataDog/java-profiler#692
  • Fix memory leak in JfrMetadata::reset() — DataDog/java-profiler#694
  • Remove legacy DirectByteBuffer-based OTEP context storage (Phase 3) — DataDog/java-profiler#696
  • Never patch the profiler's own import table — DataDog/java-profiler#721

1.65.0

Components

AI Guard

• ✨ Support DD_APM_TRACING_ENABLED=false for AI Guard traces (#11885 - @claponcet)

Application Security Management (WAF)

• ✨ Update default AppSec ruleset to release 1.18.2 (#12099 - @jandro996)
• 🐛 Stamp _dd.apm.enabled on every exported chunk when APM tracing is disabled (#12028 - @claponcet)
• 🐛 Fix WafContext TOCTOU race resurrecting orphaned native contexts (#12024 - @jandro996)
• 🐛 Fix Metaspace leak in SCA Reachability retransform and opcode injection (#12013 - @jandro996)
• ✨ Report agentic onboarding marker in configuration telemetry (RFC-1113) (#11983 - @christophe-papazian)
• 🐛 Bound multipart text/plain field materialization in Jersey AppSec (#11944 - @jandro996)
• 🐛 Update libddwaf-java to 17.4.0 (#11943 - @jandro996)
• ✨ AppSec Lambda: Analyze HTTP response (#11750 - @claponcet)
• ✨ Add server.request.body.files_content AppSec address for Undertow and Play (#11726 - @jandro996)
• ✨ Add server.request.body.files_content AppSec address for Akka HTTP (#11725 - @jandro996)
• ✨ Add server.request.body.files_content AppSec address for Vert.x 3/4/5 (#11724 - @jandro996)
• ✨ Add server.request.body.files_content AppSec address for Jetty (#11706 - @jandro996)

Build & Tooling

• ✨ Update JMXFetch to 0.52.1 (#12121 - @sarahchen6)

Continuous Integration Visibility

• 🐛 Restore JUnit4 test ordering on Gradle 9.3+ (#12098 - @daniel-mohedano)
• ✨ Add ci.pipeline.display_name tag for Buildkite pipelines (#12079 - @gbrandrea)
• ✨ Allow Gradle verification of Test Optimization dependencies (#12077 - @daniel-mohedano)
• ✨ Improve codeowners lookup performance (#12058 - @daniel-mohedano)
• 🐛 Fix GitLab codeowners exclusion handling (#12021 - @daniel-mohedano)
• 🐛 Stop retrying aborted Karate scenarios (#12015 - @daniel-mohedano)
• 🐛 Finish Karate tests after scenario post-processing (#12014 - @daniel-mohedano)
• 🐛 Avoid retrying untracked Karate scenarios (#12012 - @daniel-mohedano)
• ✨ Improve GitLab codeowners parsing (#11999 - @daniel-mohedano)
• ✨ Add code coverage report flags (#11981 - @juan-fernandez)
• ✨ Memoize per-test class analysis in a module-wide cache (#11969 - @daniel-mohedano)
• ✨ Add test reporting for Karate v2 (#11923 - @daniel-mohedano)
• ✨ Avoid error logging when parsing optional sections in codeowners file (#11881 - @daniel-mohedano)
• ✨ Add AGP usage visibility for Gradle projects (#11869 - @daniel-mohedano)
• ✨ Add Android/Robolectric visibility to Test Optimization spans (#11852 - @daniel-mohedano)
• ✨ Avoid final field modifications in Cucumber and Spock instrumentations (#11851 - @daniel-mohedano)
• 🐛 Fix Cucumber test retries running under Suite (#11850 - @daniel-mohedano)
• ✨ Avoid final field modification in JUnit5 instrumentation (#11752 - @daniel-mohedano)

Data Streams Monitoring

• 🧹 Remove full schema collection from Kafka, Avro, and Protobuf instrumentation (#12038 - @piochelepiotr)
• 🐛 Tag consumer spans with the pathway hash on every data streams checkpoint (#11808 - @ericfirth)

Dynamic Instrumentation

• 🐛 Avoid calling synchronized wrappers when capturing (#12106 - @tylfin)
• 🐛 Fix initialization of samplers for probe file (#12067 - @jpbempel)
• 🐛 Update record detection for fixed JDK (#12031 - @jpbempel)
• 🐛 Update MethodParameters detection for fixed JDK (#12025 - @jpbempel)
• 🐛 Fix NullPointerException in Exception Replay (#11976 - @jpbempel)
• 🐛 Avoid false warning when no stacktrace (#11971 - @jpbempel)
• ✨ Hoist local vars for exception probes (#11964 - @jpbempel)
• 🐛 Fix instrumenting metric probes for Kotlin (#11890 - @jpbempel)

ML Observability (LLMObs)

• ✨ Add LLMObs Tool Definitions API (#12072 - @sabrenner)

Metrics

• ✨ Emit additional metric tags on the OTLP stats export path (client-side stats) (#12069 - @dougqh)
• ✨ Add span-derived primary tags (CSS v1.3.0) (#11402 - @dougqh)

OpenFeature

• ✨ Add agentless Feature Flagging configuration source (#11892 - @leoromanovsky)
• ✨ Add experimental, gated feature-flag span enrichment (#11658 - @leoromanovsky)

Platform

• ✨ Improve JSON component (#12086 - @PerfectSlayer)

Profiling

• 🐛 Update java-profiler (ddprof) to 1.48.1 (#12133 - @kaahos)
  • Gracefully disable the profiler instead of crashing when libgcc_s.so.1 is unavailable by @zhengyu123 in DataDog/java-profiler#689
  • Avoid retaining invalid HotSpot VM method pointers during stack walking and JFR serialization by @jbachorik in DataDog/java-profiler#699
• 🐛 Update java-profiler (ddprof) to 1.48.0 (#12076 - @kaahos)
  • Add native malloc/socket stack capture with profiler-hook frame skipping by @jbachorik in DataDog/java-profiler#648
  • Add categorized native-memory accounting by @rkennke in DataDog/java-profiler#669
  • Set up signal handlers before initializing VMStructs and protect VMFlag::name() calls by @zhengyu123 in DataDog/java-profiler#674
  • Make CodeCache memoryUsage() accurate and live by @rkennke in DataDog/java-profiler#677
  • Report the actual runtime-stubs cache size by @rkennke in DataDog/java-profiler#681
  • Use sigsetjmp/siglongjmp instead of setjmp/longjmp by @zhengyu123 in DataDog/java-profiler#686
• 🐛 Update java-profiler (ddprof) to 1.47.0 (#12076 - @rkennke)
  • Add all-native OTEL context write API — Phase 1 (PROF-15271) by @rkennke in DataDog/java-profiler#631
  • Guard line-number-table copy against stale jmethodID by @jbachorik in DataDog/java-profiler#647
  • Prevent duplicate trace_id after CallTraceHashTable expansion by @jbachorik in DataDog/java-profiler#659
  • Fix native OOM from unbounded CallTraceHashTable expansion by @jbachorik in DataDog/java-profiler#670
• 🔐 Address CVE-2026-59949 by upgrading LZ4 to v1.11.1 (#12091 - @vandonr)
• 🐛 Migrate context bridge to the all-native DDProf API (Phase 2) (#11899 - @rkennke)

Testing

• ✨ Add APM test agent trace decoding (#12089 - @PerfectSlayer)
• ✨ Add decoded span assert rules (#12088 - @PerfectSlayer)
• ✨ Avoid null collection on decoded spans (#12087 - @PerfectSlayer)
• ✨ Add test agent session token support (#12050 - @PerfectSlayer)

Trace context propagation

• 🐛 Decouple W3C lastParentId from shared propagation-tag mutation (#11702 - @dougqh)

Tracer core

• ✨ Fix propagation tag serialization for trace chunks (#12096 - @sarahchen6)
• ✨ Report OTLP export status in tracer startup log (#12062 - @bm1549)
• 🐛 Fix lookup-then-read race in GenerationalUtf8Cache.getUtf8 (#11917 - @dougqh)
• ✨ Wire per-component cardinality limits from Config (#11732 - @dougqh)

Tracer internal logging

• 💡 Enable secure random ID generation for AWS Lambda MicroVM environments (#11622 - @litianningdatadog)

Instrumentations

Apache Spark instrumentation

• ✨ Add Spark 4.0.0 test suite on JDK 17 to spark_2.13 (#12049 - @aboitreaud)
• ✨ Mix Databricks repair attempt into Spark parent trace id (#12022 - @aboitreaud)

Core Java language instrumentation

• ⚡🧹 Replace ThreadPoolExecutor context-store with ClassValue cache (#12105 - @mcculls)

Kafka instrumentation

• 🧹 Remove full schema collection from Kafka, Avro, and Protobuf instrumentation (#12038 - @piochelepiotr)
• 🐛 Fix NoSuchElementException from unchecked Optional.get() in Kafka consumer instrumentation (#12007 - @piochelepiotr)

Netty instrumentation

• 🐛 Keep Netty 4.1 server spans open until streaming responses complete (#11941 - @ygree)
• 🐛 Fix Netty 4.1 tracing for pipelined HTTP/1.1 responses (#11937 - @ygree)

OpenTelemetry instrumentation

• ✨ Telemetry for OTLP traces/metrics/logs (#12057 - @mcculls)
• ✨⚡ Limit growth of OTLP trace buffer (#12052 - @mcculls)
• ✨ Support OTLP export over "http/json" for traces, metrics, and logs (#12018 - @mcculls)

Protocol Buffer instrumentation

• 🧹 Remove full schema collection from Kafka, Avro, and Protobuf instrumentation (#12038 - @piochelepiotr)

RabbitMQ instrumentation

• 🐛 Use routing key as DSM topic for RabbitMQ default-exchange publishes (#11805 - @ericfirth)

Reactor instrumentation

• ✨ Support W3C baggage propagation across reactor-netty connect hand-off (#12027 - @mcculls)
• 🐛 Fix multicast context propagation race in reactor/reactive-streams (#11825 - @amarziali)

Spring instrumentation

• 🐛 Tag @Async and @Scheduled spring-scheduling spans as errored on exception (#12071 - @ValentinZakharov)

All other instrumentations

• ✨ Support W3C baggage propagation across various async instrumentation (#12033 - @mcculls)
• 🐛 Add missing throwable suppression on instrumentation advice (#12019 - @PerfectSlayer)
• 🐛 Fix Undertow versions > 2.2.40, 2.3.25 and 2.4.0 (#11844 - @vandonr)

1.64.2

Components

Application Security Management (WAF)

• 🐛 Fix Metaspace leak in SCA Reachability retransform and opcode injection (#12075 - @claponcet)
• 🐛 Stamp _dd.apm.enabled on every exported chunk when APM tracing is disabled (#12068 - @claponcet)

Profiling

• ✨ Bump java-profiler (ddprof) to 1.47.0 (#12065 - @jbachorik)

Realtime User Monitoring

• 🐛 Fix RUM injection for offset response writes (#12064 - @PerfectSlayer)

Instrumentations

Kafka instrumentation

• 🐛 Fix NoSuchElementException from unchecked Optional.get() in Kafka consumer instrumentation (#12066 - @PerfectSlayer)

1.64.1

Components

Tracer core

• 🐛 Guard InternalTagAdder against null prebuilt tagmap entry (#11962 - @amarziali)

1.64.0

Note

When the sampling.priority tag has a positive value we now translate that to forceKeep to ensure the trace segment is kept regardless of any previous sampling decision

Components

Application Security Management (IAST)

• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨⚡ Improve performance of regexps in IAST and query obfuscator (#11649 - @manuel-alvarez-alvarez)
• 🐛 Fix IAST JDBC instrumentation when getMetadata throws non-SQLException (#11420 - @smola)

Application Security Management (WAF)

• 🐛 Fix unbounded multipart form-field buffering in Jetty 8 PartHelper (#11876 - @jandro996)
• ✨ Add request_excluded tag to appsec.waf.requests metric (#11744 - @jandro996)
• 🐛 Fix rasp.rule.match metric always reporting block:N/A (#11723 - @jandro996)
• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨ Fix waf.updates metric success:N/A tag from sequential counter metric values (#11604 - @jandro996)
• ✨ Implement SCA Reachability runtime detection: report vulnerable classes and callsites via telemetry (#11352 - @jandro996)
• ✨ Add server.request.body.filenames AppSec address for Vert.x 3.4, 4.0 and 5.0 (#11268 - @jandro996)
• ✨ Add server.request.body.filenames support for Jetty (#10988 - @jandro996)

Build & Tooling

• ✨🔐 Forbid OkHostnameVerifier to prevent accidental use, CVE-2021-0341 (#11694 - @bric3)

Configuration at Runtime

• ✨ Avoid logging stack trace during expected remote-config exception (#11328 - @sarahchen6)

Configuration

• ✨ Add OTLP Trace Metrics Configs (#11653 - @mhlidd)
• ✨⚡ Optimize collection creation and memory allocation of supported config (#11616 - @PerfectSlayer)

Continuous Integration Visibility

• ✨ Avoid final field modifications in Karate instrumentation (#11753 - @daniel-mohedano)
• ✨ Avoid final field modification in Weaver instrumentation (#11728 - @daniel-mohedano)
• 🐛 Disable invokedynamic bytecode for legacy Gradle instrumentation (#11618 - @daniel-mohedano)
• ✨ Support Maven generateTestClasspath lookup on 3.6.0+ (#11577 - @daniel-mohedano)
• ✨ Truncate CI Visibility meta tag values (#11485 - @juan-fernandez)

Crash Tracking

• 💡 Include JVM distribution details in crash report (#11715 - @gyuheon0h)
• ✨ Improve crash report message when signal is absent (#11681 - @amarziali)
• 🐛 Avoid crashtracking temp-script path hijack (#11613 - @jbachorik)

Data Streams Monitoring

• ✨ Update EventBridge DSM topic naming (#11603 - @jeastham1993)

Dynamic Instrumentation

• 🐛 Fix Kotlin Debug source mapping (#11803 - @jpbempel)
• ✨ Add evaluation timeout for expressions (#11741 - @jpbempel)
• 🐛 Fix Spring MethodParameters detection (#11647 - @jpbempel)
• 🐛 Fix sampling log probe with capture expressions (#11571 - @jpbempel)

Metrics

• ✨🧹 Update AggregateEntry to Split OK/Error Latencies (#11719 - @mhlidd)
• ✨⚡ Lazy-allocate error latency histogram on AggregateEntry (#11478 - @dougqh)
• ✨⚡ Introduce per-component / tag cardinality limits in client-side stats (#11387 - @dougqh)

OpenFeature

• 🐛 Use agent OTel metrics for OpenFeature evaluations (#11745 - @sameerank)
• 🐛 Skip malformed FFE flags during config parsing (#11534 - @leoromanovsky)
• 🧹 Use canonical FFE fixtures (#11355 - @leoromanovsky)

Profiling

• 🐛 Bump ddprof to 1.46.0 (#11858 - @jbachorik)
  • Add alternative TLS implementation to fix a crash on musl/aarch64 by @zhengyu123 in DataDog/java-profiler#600
  • Don't preload jmethodIDs when cstack=vm with hotspot JVM by @zhengyu123 in DataDog/java-profiler#549
  • Scope OTEL ThreadContext storage to the carrier thread (PROF-15271) by @rkennke in DataDog/java-profiler#625
  • Close two TOCTOU races between SIGPROF handler and JFR lifecycle by @r1viollet in DataDog/java-profiler#614
• 🐛 Request carrier-scoped ddprof context storage on JDK 21+ (#11826 - @rkennke)
• 🐛 Restore app context attributes wiped by ddprof setContext on span activation (#11646 - @jbachorik)
• 🐛 Publish thread context attribute keys at process start (#11558 - @ivoanjo)
• ✨ Add force jmethodID experimental feature support (#11549 - @zhengyu123)

Telemetry

• ✨ Migrate SCA Reachability to method-level symbol database (sca-reachability-symbols) (#11614 - @jandro996)
• ✨ Fix waf.updates metric success:N/A tag from sequential counter metric values (#11604 - @jandro996)
• ✨ Implement SCA Reachability runtime detection: report vulnerable classes and callsites via telemetry (#11352 - @jandro996)

Trace context propagation

• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨ Implement an Org Propagation Guard mechanism (#11289 - @amarziali)

Tracer core

• ✨ Introduce api-management inferred proxy spans to Azure (#11804 - @TophrC-dd)
• 🧹 Migrate simple try-with-resources use of AgentScope to ContextScope (#11763 - @mcculls)
• ⚡🧹 Overhaul set benchmarks: split Immutable / SingleThreaded, add Set.copyOf (#11721 - @dougqh)
• ✨ Wire -Pjmh.includes and -PtestJvm into internal-api JMH config (#11703 - @dougqh)
• 🐛🔐 Fix AppSec/IAST context dropped when trace.propagation.behavior.extract=ignore/restart (#11666 - @dougqh)
• ✨ Update jnr-unixsocket dependency to 0.38.25 (#11615 - @mcculls)
• ✨ Update dd-instrument-java dependency to v0.0.4 (#11567 - @mcculls)
• ✨ Update byte-buddy dependency to 1.18.10 (#11565 - @mcculls)
• ✨⚡ Pre-construct TagMap.Entry objects in InternalTagsAdder (#11555 - @dougqh)
• 🐛 Fix JDBC URL parsing crash when no port specified (#11550 - @ValentinZakharov)
• 🐛 Build DatadogClassLoader resource URL from agent jar URL, not OS path (#11480 - @dougqh)
• ✨ Include advice class name in instrumentation exception handler log (#11457 - @amarziali)
• ✨⚡ Use TagMap.set in CoreTracer (#11106 - @dougqh)
• 📖⚡ List iteration benchmark (#10888 - @dougqh)
• 📖⚡ Atomics benchmark (#10862 - @dougqh)
• ✨⚡ Adding lighter String processing methods to Strings (#10640 - @dougqh)

Tracer public API

• 💡⚠️ Translate a sampling.priority tag set to a positive value to forceKeep (#11621 - @mcculls)
• 🐛 Fix DDTraceId/DD64bTraceId class-initialization deadlock (#11509 - @bm1549)

Instrumentations

Apache HttpComponents

• 🐛🔐 Fix private headers leaking to a different origin on redirect (#11671 - @vandonr)

Apache Spark instrumentation

• ✨ Inject ol_env into ol spark run tags (#11786 - @ajyu)

Armeria Instrumentation

• 🐛 Fix Ameria 1.40 instrumentation compatibility (#11827 - @vandonr)
• ✨ Ignore capturing connection continuation for armeria (#11657 - @amarziali)

AWS Lambda instrumentation

• 🐛 Don't add empty headers on lambda completion (#11551 - @amarziali)

AWS SDK instrumentation

• 🐛 Fix message handler spans appear disconnected from the incoming SQS trace (#11511 - @ygree)

Azure Functions instrumentation

• ✨ Introduces the api-management inferred proxy spans to Azure (#11804 - @TophrC-dd)

Core Java language instrumentation

• ✨ Avoid leaking continuations on jdk-http-client (#11557 - @amarziali)

Eclipse Vert.x instrumentation

• ✨ Add support for Vert.x 5.1 in tracing (#11655 - @vandonr)

JAX-RS instrumentation

• ✨ Avoid leaking continuations on internal rxjava schedulers (#11540 - @amarziali)

JDBC instrumentation

• 🐛🔐 Fix password leak from JDBC URL (#11568 - @ValentinZakharov)
• 🐛 Fix JDBC URL parsing crash when no port specified (#11550 - @ValentinZakharov)
• 🐛 Fix IAST JDBC instrumentation when getMetadata throws non-SQLException (#11420 - @smola)

Kafka instrumentation

• 🐛 Add safe extraction for malformed base64 kafka headers (#11472 - @amarziali)

Lettuce instrumentation

• 🐛 Fix Redis command span decoration to resolve node connection for master-replica providers in Lettuce 5+ (#11819 - @ygree)
• 🐛 Fine tune async propagation for lettuce 5 (#11607 - @amarziali)

Netty instrumentation

• ✨ Do not capture continuation on netty GlobalEventExecutor static init (#11654 - @amarziali)
• 🐛 Fix continuation leakage on netty http2 client pipeline (#11595 - @amarziali)

Reactor instrumentation

• 🧹 Make reactor-core / reactive-streams context propagation Context-first (#11578 - @amarziali)

Servlet instrumentation

• 🐛 Fix StackOverflowError in caused by missing re-entrancy guard (#11838 - @ValentinZakharov)
• 🐛 Tolerate servlet 2.x wrappers used on servlet-3 classpath for isAsyncStarted calls (#11542 - @amarziali)

Spring instrumentation

• ✨ Drop the duplicate handler resolution in Spring Web 6 instrumentation (#11533 - @amarziali)
• 🐛 Build DatadogClassLoader resource URL from agent jar URL, not OS path (#11480 - @dougqh)

All other instrumentations

• ✨ Add RxJava 3 instrumentation (#11849 - @ValentinZakharov)
• ✨ Support proper async propagation for GAX retrying futures (#11629 - @amarziali)

1.63.2

Note

This release contains the same changes as in release 1.63.1. Release 1.63.1 is not flawed in any way.

Components

Application Security Management (IAST)

• ✨⚡ Improve performance of regexps in IAST and query obfuscator (#11710 - @manuel-alvarez-alvarez)

Continuous Integration Visibility

• ✨ Support Maven generateTestClasspath lookup on 3.6.0+ (#11634 - @daniel-mohedano)

Dynamic Instrumentation

• 🐛 Fix sampling log probe with capture expressions (#11628 - @jpbempel)

OpenFeature

• 🐛 Skip malformed FFE flags during config parsing (#11648 - @PerfectSlayer)

1.63.1

Note

This release is missing in Docker registries.

Components

Application Security Management (IAST)

• ✨⚡ Improve performance of regexps in IAST and query obfuscator (#11710 - @manuel-alvarez-alvarez)

Continuous Integration Visibility

• ✨ Support Maven generateTestClasspath lookup on 3.6.0+ (#11634 - @daniel-mohedano)

Dynamic Instrumentation

• 🐛 Fix sampling log probe with capture expressions (#11628 - @jpbempel)

OpenFeature

• 🐛 Skip malformed FFE flags during config parsing (#11648 - @PerfectSlayer)