Tighten tracer.metrics defaults to protect tight-heap JVMs (#11500)

Tighten tracer.metrics defaults to protect tight-heap JVMs

Cut the implicit TRACER_METRICS_MAX_PENDING default from 2048 (logical) to 128
on normal heap and to 64 at Xmx < 128 MB, and the implicit
TRACER_METRICS_MAX_AGGREGATES default from 2048 to 256 at tight heap.

Why
---
The metrics inbox is an MpscArrayQueue<SpanSnapshot> sized to
maxPending * LEGACY_BATCH_SIZE (64). With one ~120 B SpanSnapshot per slot,
the prior 131072-slot default pinned ~15 MB worst-case in-flight when the
aggregator stalled.

At Xmx <= ~128 MB the G1 survivor region is too small to absorb that
footprint -- observed catastrophically at Xmx 64 MB on spring-petclinic
where the inbox overflowed young gen and triggered To-space Exhausted →
Full GC storms (0 r/s in the worst case).

New defaults bound the worst-case in-flight footprint at ~1 MB on normal
heap and ~500 KB at tight heap, comfortably below typical survivor sizes
and large enough to absorb the sub-second consumer stalls we actually see
in practice (~0.8 s of buffer at 10 K spans/s on the normal-heap default).

Customers who explicitly configure TRACER_METRICS_MAX_PENDING are
unaffected; the LEGACY_BATCH_SIZE multiplier still applies to overrides.
Only the implicit defaults shrink.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Sync tracer.metrics.max.pending metadata default to 128

The default cut from 2048 → 128 needs the matching entry in
metadata/supported-configurations.json so config-inversion tooling
and supported-configuration docs reflect the new value.

DD_TRACE_TRACER_METRICS_MAX_AGGREGATES is left at 2048: the normal-heap
default is unchanged. The metadata schema doesn't support heap-dependent
defaults, so the tight-heap branch (64 / 256) isn't representable; the
metadata reflects the normal-heap default that applies to the typical
customer (Xmx >= 128 MB).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

chore(metadata): bump DD_TRACE_TRACER_METRICS_MAX_PENDING version to B

Default changed from 2048 to 128; version field must be incremented
when the default value changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Merge branch 'master' into dougqh/css-tight-heap-defaults

Merge branch 'master' into dougqh/css-tight-heap-defaults

Merge branch 'master' into dougqh/css-tight-heap-defaults

Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>

Author: dougqh

internal-api/src/main/java/datadog/trace/api/Config.java

@@ -2191,22 +2191,43 @@ private Config(final ConfigProvider configProvider, final InstrumenterConfig ins
         configProvider.getBoolean(TRACE_STATS_COMPUTATION_IGNORE_AGENT_VERSION, false);
     tracerMetricsBufferingEnabled =
         configProvider.getBoolean(TRACER_METRICS_BUFFERING_ENABLED, false);
-    tracerMetricsMaxAggregates = configProvider.getInteger(TRACER_METRICS_MAX_AGGREGATES, 2048);
+    // The metrics inbox is an MpscArrayQueue<SpanSnapshot>; each saturated slot holds one
+    // ~120 B SpanSnapshot. The historical default TRACER_METRICS_MAX_PENDING=2048 (logical) *
+    // LEGACY_BATCH_SIZE=64 = 131072 slots was sized for the prior conflating-Batch model where
+    // slot memory was only realized under burst; with one snapshot per slot, the worst-case
+    // in-flight footprint is ~15 MB. At Xmx <= ~128 MB the G1 survivor region is too small to
+    // absorb that footprint when the aggregator stalls -- observed catastrophically at Xmx64m
+    // petclinic where SpanSnapshots overflow young gen and trigger To-space Exhausted -> Full
+    // GC storms (0 r/s in the worst case).
+    //
+    // Cut the default accordingly:
+    //   - normal heap: 128 logical * 64 = 8192 slots, ~1 MB worst-case in-flight. ~0.8 s of
+    //     buffer at 10K spans/s, well above typical GC pause windows.
+    //   - tight heap (Xmx < 128 MB): 64 logical * 64 = 4096 slots, ~500 KB worst case.
+    //
+    // Customers who explicitly configured TRACER_METRICS_MAX_PENDING keep their value (the
+    // LEGACY_BATCH_SIZE multiplier still applies to it) -- only the implicit default shrinks.
+    final boolean tightHeap = Runtime.getRuntime().maxMemory() < 128L * 1024 * 1024;
+    final int defaultMaxAggregates = tightHeap ? 256 : 2048;
+    final int defaultMaxPending = tightHeap ? 64 : 128;
+
+    tracerMetricsMaxAggregates =
+        configProvider.getInteger(TRACER_METRICS_MAX_AGGREGATES, defaultMaxAggregates);
     /*
      * TRACER_METRICS_MAX_PENDING historically counted conflating Batch slots (~64 spans per batch
      * via Batch.MAX_BATCH_SIZE). The inbox now holds 1 SpanSnapshot per metrics-eligible span, so
      * we multiply the configured value by the legacy batch size to preserve the effective
-     * span-throughput capacity of the prior default *and* of any existing customer override
-     * (e.g. a configured 4096 still means "~262144 spans before drops", same as before). ~100 B
-     * per SpanSnapshot * 131072 ≈ 13 MB worst-case heap floor at the default.
+     * span-throughput capacity for any existing customer override (e.g. a configured 4096 still
+     * means "~262144 spans before drops", same as before).
      *
      * Long-promote the multiplication and clamp to MAX_SAFE_ARRAY_SIZE so an absurd customer
      * override (>= ~33M) can't silently wrap to a negative int. MAX_SAFE_ARRAY_SIZE sits a few
      * bytes below Integer.MAX_VALUE because the JVM reserves header slack on array allocations;
      * see java.util.ArraysSupport.SOFT_MAX_ARRAY_LENGTH for the same convention.
      */
     long requestedMaxPending =
-        (long) configProvider.getInteger(TRACER_METRICS_MAX_PENDING, 2048) * LEGACY_BATCH_SIZE;
+        (long) configProvider.getInteger(TRACER_METRICS_MAX_PENDING, defaultMaxPending)
+            * LEGACY_BATCH_SIZE;
     tracerMetricsMaxPending = (int) Math.min(requestedMaxPending, MAX_SAFE_ARRAY_SIZE);
 
     reportHostName =

metadata/supported-configurations.json

@@ -10915,9 +10915,9 @@
     ],
     "DD_TRACE_TRACER_METRICS_MAX_PENDING": [
       {
-        "version": "A",
+        "version": "B",
         "type": "int",
-        "default": "2048",
+        "default": "128",
         "aliases": []
       }
     ],