fix(llmobs): render BaseMessage in langgraph workflow I/O - #8097
Conversation
The langgraph plugin had its own formatIO that only recursed plain
Object/Array, so BaseMessage instances (HumanMessage, AIMessage, …)
fell through to JSON.stringify and serialized the full class shape
(additional_kwargs, response_metadata, id, tool_call_id, …) into the
workflow span's Input/Output panels.
Extract the langchain handler's formatIO/getContentFromMessage/getRole
into a shared util under llmobs/plugins/shared/messages.js and use it
from both plugins, so workflow spans and child chat_model spans render
messages consistently as { content, role }.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Overall package sizeSelf size: 5.56 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.1 | 82.56 kB | 817.39 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #8097 +/- ##
==========================================
+ Coverage 73.70% 73.79% +0.08%
==========================================
Files 783 784 +1
Lines 36369 36369
==========================================
+ Hits 26805 26837 +32
+ Misses 9564 9532 -32 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
✨ Fix all issues with BitsAI or with Cursor
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4096f85158
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } | ||
|
|
||
| if (Array.isArray(data)) { | ||
| return data.map(item => formatIO(item)) |
There was a problem hiding this comment.
The new shared formatIO routes every non-plain object to getContentFromMessage, which creates { content: message.content || '' }. In LangGraph workflows, state often contains class instances that are not chat messages (for example Document-like or user-defined objects without a content property), and these now get reduced to {"content":""} instead of preserving their serialized data as before. This is a regression in workflow I/O tagging because it drops meaningful state content for any non-BaseMessage object.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 5503957. formatIO now only routes duck-typed BaseMessage instances (typeof data._getType === 'function' || typeof data.getType === 'function') to getContentFromMessage. Any other class instance falls back to JSON.stringify(data) — matching the pre-#8097 LangGraph behavior so Document-like or user-defined objects without a content property retain their serialized shape instead of collapsing to {content: ''}. Added a regression unit test in packages/dd-trace/test/llmobs/plugins/shared/messages.spec.js ("preserves non-message class instances via JSON.stringify...").
BenchmarksBenchmark execution time: 2026-04-27 19:14:13 Comparing candidate commit cfd4e5e in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1344 metrics, 100 unstable metrics. |
Addresses Codex review on PR #8097. The new shared formatIO previously routed every non-plain-object/non-array through getContentFromMessage, collapsing class instances without a `content` property (e.g. LangChain Document) to `{ content: '' }` and dropping meaningful state data in LangGraph workflow I/O. Only duck-typed BaseMessage instances (have `_getType`/`getType`) now produce `{ content, role }`; other class instances fall back to JSON.stringify, matching the pre-#8097 LangGraph behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sabrenner
left a comment
sabrenner
left a comment
There was a problem hiding this comment.
just some cleanup 🧹 things but lgtm!
…hain Address PR #8097 review from @sabrenner: - Remove formatIO / getContentFromMessage / getRole wrapper methods on LangChainLLMObsHandler; chain.js, embedding.js, vectorstore.js, and chat_model.js now import the utility directly. - Move plugins/shared/messages.js into plugins/langchain/messages.js so there's no shared/ dir; LangGraph imports from ../langchain/messages. - Drop the shared/ unit test file — coverage exists via the LangChain and LangGraph integration specs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(llmobs): render BaseMessage in langgraph workflow I/O (#8096) Extract the langchain handler's formatIO/getContentFromMessage/getRole into a shared util under and use it for both plugins, so workflow spans and child chat_model spans render messages consistently as { content, role }.
* fix(llmobs): render BaseMessage in langgraph workflow I/O (#8096) Extract the langchain handler's formatIO/getContentFromMessage/getRole into a shared util under and use it for both plugins, so workflow spans and child chat_model spans render messages consistently as { content, role }.
2 participants
What does this PR do?
Fixes #8096. The LangGraph LLM Obs plugin's workflow span now renders
BaseMessageinstances (HumanMessage,AIMessage,SystemMessage, …) as{ content, role }objects, matching the LangChain plugin's childchat_modelspans — instead of dumping the full class withadditional_kwargs,response_metadata,id,tool_call_id, etc. into the Input/Output panels.Motivation
The LangGraph plugin (added in #7567) shipped its own
formatIOthat only recursed plainObject/Array. SinceBaseMessagesubclasses haveconstructor.nameofHumanMessage/AIMessage/ etc., they fell through toJSON.stringify(data)and serialized the entire class shape. This made the workflow span emitted bycreateAgent(...).invoke(...)/Pregel.streamessentially unreadable, even though the child chat_model and tool spans rendered correctly.What changed
ROLE_MAPPINGS,getRole,getContentFromMessage, andformatIOout ofpackages/dd-trace/src/llmobs/plugins/langchain/handlers/index.jsinto a new shared module atpackages/dd-trace/src/llmobs/plugins/shared/messages.js.this.formatIO/this.getContentFromMessage/this.getRolepreserved as thin wrappers sochain.js,chat_model.js,embedding.js,vectorstore.jskeep working unchanged).packages/dd-trace/src/llmobs/plugins/langgraph/index.jsto use the sharedformatIO, which now falls through togetContentFromMessagefor non-plain-object values — handlingBaseMessagethe same way LangChain does.Testing
packages/dd-trace/test/llmobs/plugins/shared/messages.spec.jscoveringgetRole(explicit role,_getType,getType),getContentFromMessage(string passthrough, BaseMessage →{ content, role }, missing content default), andformatIO(null / primitive / plain object / array / BaseMessage / nested LangGraph-shape state).packages/dd-trace/test/llmobs/plugins/langgraph/index.spec.jsthat drives a Pregel graph with realHumanMessage/AIMessageinstances and asserts the workflow span'sinput.value/output.valuerender as{ content, role }.langgraphjob in.github/workflows/llmobs.yml.Additional Notes
This is a strictly additive refactor: the LangChain handler's observable behavior is unchanged aside from
formatIO(null)/formatIO(undefined)/formatIO(primitive)at the root (previously crashed or returned{ content: '' }, now returns''or the primitive). Existing LangChain call sites always pass objects, so no test fixtures need to change.🤖 Generated with Claude Code