Skip to content

Commit 41fffe8

Browse files
dougqhdevflow.devflow-routing-intake
dougqh
and
devflow.devflow-routing-intake
authored
Wire per-component cardinality limits from Config (#11732)
Wire per-component cardinality limits from Config per Cardinality Limits RFC Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Raise RESOURCE cardinality limit to 1024 to avoid premature collapse Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Apply spotless formatting Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Apply spotless formatting to Config.java Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Fix supported-configurations.json: use int not integer for cardinality limits Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Align cardinality config with .NET: rename peer_tags, raise http_endpoint, add table alias - Rename DD_TRACE_STATS_PEER_TAG_CARDINALITY_LIMIT -> DD_TRACE_STATS_PEER_TAGS_CARDINALITY_LIMIT to match .NET naming (plural) - Raise DD_TRACE_STATS_HTTP_ENDPOINT_CARDINALITY_LIMIT default 64 -> 512 to match .NET - Add DD_TRACE_STATS_CARDINALITY_LIMIT as the canonical name for the aggregate table cap, aliasing the older DD_TRACE_TRACER_METRICS_MAX_AGGREGATES - Add explicit string constants for all 10 per-field cardinality limit keys in GeneralConfig so the config registry validator can find them Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Drop 7 Java-only per-field cardinality limit config keys SERVICE, OPERATION, SERVICE_SOURCE, TYPE, SPAN_KIND, HTTP_METHOD, and GRPC_STATUS_CODE have naturally low cardinality -- the aggregate table cap (DD_TRACE_STATS_CARDINALITY_LIMIT) is sufficient backstop. Aligns the public config surface with .NET (resource, http_endpoint, peer_tags, table cap). Hardcoded defaults in MetricCardinalityLimits remain; config knobs can be added later if a customer needs to tune them. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Merge branch 'master' into dougqh/cardinality-limits-config Fix fixed-limit fields to bypass Config lookup, avoiding unsupported config key detection Fields with hardcoded limits (service, operation, type, span_kind, http_method, grpc_status_code, service_source) now pass their MetricCardinalityLimits constant directly to PropertyCardinalityHandler instead of going through Config.getTraceStatsCardinalityLimit. This prevents the test harness from flagging DD_TRACE_STATS_SERVICE_CARDINALITY_LIMIT etc. as unsupported config keys. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Remove aliases from DD_TRACE_STATS_CARDINALITY_LIMIT registry entry The central registry has no alias for this key; the local entry must match. The code-level alias (Config.java reads DD_TRACE_TRACER_METRICS_MAX_AGGREGATES as a fallback) is unaffected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Bump DD_TRACE_STATS_CARDINALITY_LIMIT to version B with legacy alias Version A must match the central registry exactly (no aliases). The DD_TRACE_TRACER_METRICS_MAX_AGGREGATES alias is Java-specific, so it belongs in version B — matching the same pattern as DD_TRACE_STATS_COMPUTATION_ENABLED aliasing DD_TRACE_TRACER_METRICS_ENABLED. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Merge branch 'master' into dougqh/cardinality-limits-config Merge branch 'master' into dougqh/cardinality-limits-config Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
1 parent eaabb65 commit 41fffe8

7 files changed

Lines changed: 88 additions & 32 deletions

File tree

‎dd-trace-api/src/main/java/datadog/trace/api/config/GeneralConfig.java‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,14 @@ public final class GeneralConfig {
7070
public static final String TRACE_STATS_COMPUTATION_IGNORE_AGENT_VERSION =
7171
"trace.stats.computation.ignore.agent.version";
7272

73+
public static final String TRACE_STATS_CARDINALITY_LIMIT = "trace.stats.cardinality.limit";
74+
public static final String TRACE_STATS_RESOURCE_CARDINALITY_LIMIT =
75+
"trace.stats.resource.cardinality.limit";
76+
public static final String TRACE_STATS_HTTP_ENDPOINT_CARDINALITY_LIMIT =
77+
"trace.stats.http_endpoint.cardinality.limit";
78+
public static final String TRACE_STATS_PEER_TAGS_CARDINALITY_LIMIT =
79+
"trace.stats.peer_tags.cardinality.limit";
80+
7381
public static final String TRACE_OTEL_SEMANTICS_ENABLED = "trace.otel.semantics.enabled";
7482

7583
public static final String TRACER_METRICS_ENABLED = "trace.tracer.metrics.enabled";
@@ -79,6 +87,7 @@ public final class GeneralConfig {
7987
public static final String TRACER_METRICS_MAX_PENDING = "trace.tracer.metrics.max.pending";
8088
public static final String TRACER_METRICS_IGNORED_RESOURCES =
8189
"trace.tracer.metrics.ignored.resources";
90+
8291
public static final String AZURE_APP_SERVICES = "azure.app.services";
8392
public static final String INTERNAL_EXIT_ON_FAILURE = "trace.internal.exit.on.failure";
8493

‎dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java‎

Lines changed: 20 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package datadog.trace.common.metrics;
22

33
import datadog.metrics.api.Histogram;
4+
import datadog.trace.api.Config;
45
import datadog.trace.bootstrap.instrumentation.api.UTF8BytesString;
56
import datadog.trace.util.Hashtable;
67
import datadog.trace.util.LongHashingUtils;
@@ -28,36 +29,33 @@ final class AggregateEntry extends Hashtable.Entry {
2829

2930
private static final UTF8BytesString[] EMPTY_TAGS = new UTF8BytesString[0];
3031

31-
// Sentinel substitution is disabled until per-component config is wired in a follow-up PR.
32-
// Tests that need sentinel mode should pass useBlockedSentinel=true explicitly.
33-
static final boolean LIMITS_ENABLED = false;
34-
35-
// Per-field cardinality handlers. Limits live on MetricCardinalityLimits -- see that class for
36-
// per-field rationale.
32+
// Configurable per-field cardinality handlers — tunable via env var.
3733
static final PropertyCardinalityHandler RESOURCE_HANDLER =
38-
new PropertyCardinalityHandler("resource", MetricCardinalityLimits.RESOURCE, LIMITS_ENABLED);
34+
new PropertyCardinalityHandler(
35+
"resource",
36+
Config.get().getTraceStatsCardinalityLimit("resource", MetricCardinalityLimits.RESOURCE));
37+
static final PropertyCardinalityHandler HTTP_ENDPOINT_HANDLER =
38+
new PropertyCardinalityHandler(
39+
"http_endpoint",
40+
Config.get()
41+
.getTraceStatsCardinalityLimit(
42+
"http_endpoint", MetricCardinalityLimits.HTTP_ENDPOINT));
43+
44+
// Fixed per-field cardinality handlers — hardcoded, not user-configurable.
3945
static final PropertyCardinalityHandler SERVICE_HANDLER =
40-
new PropertyCardinalityHandler("service", MetricCardinalityLimits.SERVICE, LIMITS_ENABLED);
46+
new PropertyCardinalityHandler("service", MetricCardinalityLimits.SERVICE);
4147
static final PropertyCardinalityHandler OPERATION_HANDLER =
42-
new PropertyCardinalityHandler(
43-
"operation", MetricCardinalityLimits.OPERATION, LIMITS_ENABLED);
48+
new PropertyCardinalityHandler("operation", MetricCardinalityLimits.OPERATION);
4449
static final PropertyCardinalityHandler SERVICE_SOURCE_HANDLER =
45-
new PropertyCardinalityHandler(
46-
"service_source", MetricCardinalityLimits.SERVICE_SOURCE, LIMITS_ENABLED);
50+
new PropertyCardinalityHandler("service_source", MetricCardinalityLimits.SERVICE_SOURCE);
4751
static final PropertyCardinalityHandler TYPE_HANDLER =
48-
new PropertyCardinalityHandler("type", MetricCardinalityLimits.TYPE, LIMITS_ENABLED);
52+
new PropertyCardinalityHandler("type", MetricCardinalityLimits.TYPE);
4953
static final PropertyCardinalityHandler SPAN_KIND_HANDLER =
50-
new PropertyCardinalityHandler(
51-
"span_kind", MetricCardinalityLimits.SPAN_KIND, LIMITS_ENABLED);
54+
new PropertyCardinalityHandler("span_kind", MetricCardinalityLimits.SPAN_KIND);
5255
static final PropertyCardinalityHandler HTTP_METHOD_HANDLER =
53-
new PropertyCardinalityHandler(
54-
"http_method", MetricCardinalityLimits.HTTP_METHOD, LIMITS_ENABLED);
55-
static final PropertyCardinalityHandler HTTP_ENDPOINT_HANDLER =
56-
new PropertyCardinalityHandler(
57-
"http_endpoint", MetricCardinalityLimits.HTTP_ENDPOINT, LIMITS_ENABLED);
56+
new PropertyCardinalityHandler("http_method", MetricCardinalityLimits.HTTP_METHOD);
5857
static final PropertyCardinalityHandler GRPC_STATUS_CODE_HANDLER =
59-
new PropertyCardinalityHandler(
60-
"grpc_status_code", MetricCardinalityLimits.GRPC_STATUS_CODE, LIMITS_ENABLED);
58+
new PropertyCardinalityHandler("grpc_status_code", MetricCardinalityLimits.GRPC_STATUS_CODE);
6159

6260
// Single authoritative list used by resetCardinalityHandlers(). populateFrom() and hashOf() keep
6361
// named access for readability and to avoid per-span iteration overhead; this array ensures the

‎dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateTable.java‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -83,11 +83,11 @@ AggregateEntry findOrInsert(SpanSnapshot snapshot) {
8383
* {@code onStatsAggregateDropped}) rather than evicting an established one. Cap is sized to the
8484
* steady-state working set, so eviction is rare in the common case.
8585
*
86-
* <p>How often this fires depends on {@link AggregateEntry#LIMITS_ENABLED}. With limits enabled,
87-
* over-cap values for a given field collapse into a shared {@code blocked_by_tracer} bucket, so
88-
* the table itself rarely reaches {@code maxAggregates}. With limits disabled (the default),
89-
* over-cap values flow to distinct buckets and {@code maxAggregates} becomes the load-bearing
90-
* backstop -- the cursor-resumed scan was added specifically for this regime.
86+
* <p>With per-field cardinality limits enabled, over-cap values for a given field collapse into a
87+
* shared {@code tracer_blocked_value} bucket, so the table itself rarely reaches {@code
88+
* maxAggregates}. Without per-field limits, over-cap values flow to distinct buckets and {@code
89+
* maxAggregates} becomes the load-bearing backstop -- the cursor-resumed scan was added
90+
* specifically for this regime.
9191
*/
9292
private boolean evictOneStale() {
9393
// Two passes -- [cursor, length) then [0, cursor) -- using the half-open-range iterator. The

‎dd-trace-core/src/main/java/datadog/trace/common/metrics/MetricCardinalityLimits.java‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,10 @@ private MetricCardinalityLimits() {}
1414

1515
/**
1616
* Distinct {@code resource.name} values per cycle. Highest-cardinality field by far: DB-query
17-
* obfuscations, HTTP route templates, custom resources. Typical service: 30-200 unique.
17+
* obfuscations, HTTP route templates, custom resources. Typical service: 30-200 unique; 1024
18+
* leaves headroom for high-cardinality SQL/HTTP workloads without risking premature collapse.
1819
*/
19-
static final int RESOURCE = 256;
20+
static final int RESOURCE = 1024;
2021

2122
/**
2223
* Distinct {@code service.name} values per cycle. Local service plus downstream peer-service
@@ -57,7 +58,7 @@ private MetricCardinalityLimits() {}
5758
* Distinct {@code http.endpoint} values per cycle. Path templates -- same shape as {@code
5859
* RESOURCE} for HTTP-heavy services. Only used when {@code includeEndpointInMetrics} is enabled.
5960
*/
60-
static final int HTTP_ENDPOINT = 64;
61+
static final int HTTP_ENDPOINT = 512;
6162

6263
/**
6364
* Distinct gRPC status code values per cycle. gRPC spec defines exactly 17 codes (0-16); 24

‎dd-trace-core/src/main/java/datadog/trace/common/metrics/PeerTagSchema.java‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import static datadog.trace.api.DDTags.BASE_SERVICE;
44

55
import datadog.communication.ddagent.DDAgentFeaturesDiscovery;
6+
import datadog.trace.api.Config;
67
import datadog.trace.bootstrap.instrumentation.api.UTF8BytesString;
78
import java.util.Set;
89

@@ -74,7 +75,10 @@ static PeerTagSchema of(Set<String> names, String state) {
7475
for (int i = 0; i < names.length; i++) {
7576
this.handlers[i] =
7677
new TagCardinalityHandler(
77-
names[i], MetricCardinalityLimits.PEER_TAG_VALUE, AggregateEntry.LIMITS_ENABLED);
78+
names[i],
79+
Config.get()
80+
.getTraceStatsCardinalityLimit(
81+
"peer_tags", MetricCardinalityLimits.PEER_TAG_VALUE));
7882
}
7983
}
8084

‎internal-api/src/main/java/datadog/trace/api/Config.java‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -426,6 +426,7 @@
426426
import static datadog.trace.api.config.GeneralConfig.TRACE_DEBUG;
427427
import static datadog.trace.api.config.GeneralConfig.TRACE_LOG_LEVEL;
428428
import static datadog.trace.api.config.GeneralConfig.TRACE_OTEL_SEMANTICS_ENABLED;
429+
import static datadog.trace.api.config.GeneralConfig.TRACE_STATS_CARDINALITY_LIMIT;
429430
import static datadog.trace.api.config.GeneralConfig.TRACE_STATS_COMPUTATION_ENABLED;
430431
import static datadog.trace.api.config.GeneralConfig.TRACE_STATS_COMPUTATION_IGNORE_AGENT_VERSION;
431432
import static datadog.trace.api.config.GeneralConfig.TRACE_TAGS;
@@ -2260,7 +2261,8 @@ && isMetricsOtelEnabled()
22602261
final int defaultMaxPending = tightHeap ? 64 : 128;
22612262

22622263
tracerMetricsMaxAggregates =
2263-
configProvider.getInteger(TRACER_METRICS_MAX_AGGREGATES, defaultMaxAggregates);
2264+
configProvider.getInteger(
2265+
TRACE_STATS_CARDINALITY_LIMIT, defaultMaxAggregates, TRACER_METRICS_MAX_AGGREGATES);
22642266
/*
22652267
* TRACER_METRICS_MAX_PENDING historically counted conflating Batch slots (~64 spans per batch
22662268
* via Batch.MAX_BATCH_SIZE). The inbox now holds 1 SpanSnapshot per metrics-eligible span, so
@@ -3882,6 +3884,16 @@ public int getTraceStatsInterval() {
38823884
return traceStatsInterval;
38833885
}
38843886

3887+
/**
3888+
* Returns the per-cycle cardinality limit for the named stats field, following the RFC naming
3889+
* pattern {@code DD_TRACE_STATS_{tagName}_CARDINALITY_LIMIT} (e.g. {@code
3890+
* DD_TRACE_STATS_RESOURCE_CARDINALITY_LIMIT}). The caller supplies the default from {@code
3891+
* MetricCardinalityLimits} so per-field rationale stays co-located with the defaults.
3892+
*/
3893+
public int getTraceStatsCardinalityLimit(String tagName, int defaultLimit) {
3894+
return configProvider.getInteger("trace.stats." + tagName + ".cardinality.limit", defaultLimit);
3895+
}
3896+
38853897
public boolean isLogsInjectionEnabled() {
38863898
return logsInjectionEnabled;
38873899
}

‎metadata/supported-configurations.json‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10697,6 +10697,14 @@
1069710697
"aliases": ["DD_TRACE_TRACER_METRICS_ENABLED"]
1069810698
}
1069910699
],
10700+
"DD_TRACE_STATS_CARDINALITY_LIMIT": [
10701+
{
10702+
"version": "B",
10703+
"type": "int",
10704+
"default": "2048",
10705+
"aliases": ["DD_TRACE_TRACER_METRICS_MAX_AGGREGATES"]
10706+
}
10707+
],
1070010708
"DD_TRACE_STATS_COMPUTATION_IGNORE_AGENT_VERSION": [
1070110709
{
1070210710
"version": "A",
@@ -10705,6 +10713,30 @@
1070510713
"aliases": []
1070610714
}
1070710715
],
10716+
"DD_TRACE_STATS_RESOURCE_CARDINALITY_LIMIT": [
10717+
{
10718+
"version": "A",
10719+
"type": "int",
10720+
"default": "1024",
10721+
"aliases": []
10722+
}
10723+
],
10724+
"DD_TRACE_STATS_HTTP_ENDPOINT_CARDINALITY_LIMIT": [
10725+
{
10726+
"version": "A",
10727+
"type": "int",
10728+
"default": "512",
10729+
"aliases": []
10730+
}
10731+
],
10732+
"DD_TRACE_STATS_PEER_TAGS_CARDINALITY_LIMIT": [
10733+
{
10734+
"version": "A",
10735+
"type": "int",
10736+
"default": "512",
10737+
"aliases": []
10738+
}
10739+
],
1070810740
"DD_TRACE_STATUS404DECORATOR_ENABLED": [
1070910741
{
1071010742
"version": "A",

0 commit comments

Comments
 (0)