Repository navigation
Conversation
The gen_ai.* attributes only reached the APM span through the LLMObs span
processor, which runs on span finish and only for spans the tagger registered.
With DD_LLMOBS_ENABLED unset, `LLMObsPlugin.configure` unsubscribed the whole
LLMObs layer, so APM-only users got no searchable model, provider, application,
conversation or token usage at all.
Keep the LLMObs plugins subscribed when only LLMObs itself is off, and run a
reduced path in that mode:
start emits gen_ai.operation.name, gen_ai.request.model,
gen_ai.provider.name, gen_ai.application.name and
gen_ai.conversation.id from the span register options every
integration already builds
asyncEnd emits gen_ai.usage.* from the new `getGenAiApmUsageMetrics` hook
An integration opt-out (`tracer.use(<name>, { llmobs: false })`) still disables
the layer entirely.
The register options cover the scalars for every integration from one call site,
so no per-integration tag code is needed; usage has to be read off the response,
so it is per integration as in dd-trace-py: openai, anthropic, google-genai,
vertexai, and bedrock (from its response headers, since the body is only parsed
for the LLMObs payload). The vercel-ai, langchain and claude-agent-sdk usage
paths are entangled with payload construction and report scalars only for now.
The auxiliary subscribers that aggregate streamed chunks stay inert while LLMObs
is disabled, so the reduced path costs one small object and a few setTag calls
per LLM call and never the message-body work.
`setGenAiApmTags` moves into llmobs/gen-ai-tags.js, shared by the span processor
and the plugin base so both paths normalize a facet value the same way, and the
tagger's camelCase metric mapping becomes METRIC_KEY_ALIASES so the disabled
path can accept the spelling integrations extract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reduced path read everything from the span register options at start, so any value an integration only resolves later was missing or wrong: - Vercel AI's dd-telemetry path carries the model in the span attributes, not in the register options, so model-backed spans reported the `custom` fallback. - Anthropic and google-genai build the streamed response in their own chunk subscriber, which the reduced path had inert, so streamed calls reported no `gen_ai.usage.*` at all. - The claude-agent-sdk query span learns its session id from the resolved stream, so the root agent span never got `gen_ai.conversation.id`. - A claude-agent-sdk tool span that launches a subagent is promoted from `tool` to `agent` once the nested step records its id, which the reduced path skipped, so those wrappers reported `gen_ai.operation.name=tool`. Generalize the end hook from `getGenAiApmUsageMetrics` to `getGenAiApmEndTags`, returning any `gen_ai.*` value resolved once the operation finished, including a corrected span kind. `updateGenAiApmTags` writes only the fields present, so a late update cannot re-default a model the start path already resolved. The two chunk subscribers now accumulate token usage while LLMObs is disabled and nothing else, leaving `ctx.result` to the LLMObs path since the google-genai tracing plugin reads it for `google_genai.response.model`. Anthropic's usage merge is shared with the enabled aggregation rather than duplicated. Reported by Codex on #10300. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three gaps Codex found on #10300: - Bedrock `converse`/`converseStream` send no `x-amzn-bedrock-*-token-count` headers, so the disabled path reported every `gen_ai.usage.*` metric as zero. Read the usage the Converse response carries (`response.usage`, or the stream's `metadata.usage` event, kept the same way the other streaming integrations keep theirs), and emit no usage at all when no source reports any, since zeros are worse than nothing. - OpenAI resolves aliases to a dated model version in the response, which the enabled path writes over the request model. Report that resolved model too, so APM and LLMObs agree. - `METRIC_KEY_ALIASES` was a plain object, so a custom metric named after an `Object.prototype` member (`constructor`, `toString`) resolved to an inherited property instead of falling back to its own name, and the tagger stored it under a stringified function. The switch it replaced had no such hole. Give it and the `gen_ai.usage.*` map a null prototype: both are keyed by metric names, which `LLMObs.annotate()` exposes to users. `buildUsage` is now exported from the Bedrock utils rather than reimplemented, so both paths normalize Converse usage the same way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Master's new `jsdoc/no-restricted-syntax` rule (#10305) rejects them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keeping the LLMObs layer subscribed while LLMObs is disabled also makes Bedrock's `pendingTokenHeaders` cache live for plain APM users, which turned a LLMObs-only leak into a default one. Smithy publishes the response headers per attempt, including for errors, but its rejection path builds `ctx.response` from the error with no top-level `$metadata`, so `:complete:` consumed `undefined` and left the entry behind for every failed request carrying `x-amzn-requestid`. Resolve the request id from the error metadata as well, and cap the cache with oldest-first eviction: retried and aborted attempts publish headers no `:complete:` will ever claim, so correlating better is not sufficient on its own. Also closes two gaps in the reduced path: - The claude-agent-sdk inner `llm` span normalizes `ctx.usage` inside `setLLMObsTags`, so it reported no `gen_ai.usage.*`. The normalization moves to a shared helper the end hook uses. - A traced provider integration demotes a LangChain model span to `workflow`, and langchain-openai calling its untraced beta client for `response_format` is the case where that span is nonetheless the only model span. `handlers/ chat_model.js` corrects the kind through `changeKind` when the payload is built; the end hook now applies the same correction for APM. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…is disabled Running the integrations end to end against a live agent showed both reporting `gen_ai.usage.*` with the LLMObs layer on and nothing with it off, which leaves an `llm` span with no tokens: - The Vercel AI telemetry-event path reads usage off `ctx.result.usage`, or off the stream's `finish` chunk, which the reduced path had inert. Its metric mapping is now shared with `setTextGenerationTags`, and the chunk subscriber keeps the usage while LLMObs is disabled the way the other streaming integrations do, without touching `ctx.result`. - LangChain reads it from the result through `checkTokenUsageChatOrLLMResult`, which the end hook now calls. The helper reports zeros when the result carries no usage, so those are dropped rather than published as a measurement. With this, the `gen_ai.*` tag set an APM span carries is identical whether LLMObs is enabled or disabled, across openai, anthropic, vercel-ai and langchain. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three more findings from Codex on #10300, all reachable now that the layer stays subscribed for plain APM users: - `mergeChunkUsage` kept `message_start.message.usage` by reference and wrote the `message_delta` counts into it, so the tracer mutated a chunk the application still holds. Copy it before accumulating. The enabled aggregation shared that behaviour, so this fixes both paths. - A Bedrock request the SDK rejected for a missing or non-string `modelId` reached `parseModelId`, which assumes a string. The throw was contained by the subscriber wrapper, but it disabled the plugin for the rest of the process. - A response carrying `x-amzn-requestid` and no `x-amzn-bedrock-*-token-count` headers, which is the common error-response shape, still cached a record of undefined fields. That record read as present and published every `gen_ai.usage.*` metric as zero, so it is no longer cached. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The backend reads `gen_ai.*` tags on an APM span as user-set and builds a duplicate LLMObs span out of them. Write `_dd.llmobs.artificial_gen_ai_tags` alongside, matching dd-trace-py, so it can tell the two apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Overall package sizeSelf size: 9.03 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.5.1 | 127.66 kB | 513.61 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
🎉 All green!🧪 All tests passed 🔄 Datadog retried 1 test - 1 passed on retry 🎯 Code Coverage (details) 🔗 Commit SHA: e11cb70 | Docs | View more details | Give us feedback! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10402 +/- ##
==========================================
+ Coverage 98.74% 98.79% +0.04%
==========================================
Files 1029 1032 +3
Lines 160983 162406 +1423
Branches 74 74
==========================================
+ Hits 158964 160441 +1477
+ Misses 2019 1965 -54
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Matches both the tracer's convention for boolean tags and what dd-trace-py writes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenchmarksBenchmark execution time: 2026-09-21 17:37:51 Comparing candidate commit e11cb70 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2306 metrics, 7 unstable metrics.
|
`isLLMIntegrationEnabled` read a plugin manager captured when the module first loaded. A tracer rebuilt after that point left the lookup pointing at a manager with no plugins, so the LangChain model span was never demoted and both it and the provider span reported `llm`, double-counting tokens. Read the manager off the tracer that owns the plugin instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ning tracer" This reverts commit 9689c0a.
…wning tracer" This reverts commit 6b33ee5.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Relands #10261 (reverted in #10381), plus the gen_ai APM tags for the LLMObs-disabled path, plus the marker tag that fixes the revert cause:
_dd.llmobs.artificial_gen_ai_tags = "True".Motivation
Without the marker the backend reads tracer-written
gen_ai.*tags as user-set and builds a duplicate LLMObs span out of the APM span. Port of DataDog/dd-trace-py#20411.Additional Notes
Unlike py, the marker is written only when a
gen_ai.*tag is actually set, since JS has a separate update path for values an integration resolves after the span starts.🤖 Generated with Claude Code