feat(llmobs): add bedrock agents tracing #13439
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 279 ± 3 ms. The average import time from base is: 281 ± 4 ms. The import time difference between this PR and base is: -1.6 ± 0.2 ms. Import time breakdownThe following import paths have shrunk:
|
BenchmarksBenchmark execution time: 2025-06-16 21:37:25 Comparing candidate commit f6a4841 in PR branch Found 0 performance improvements and 3 performance regressions! Performance is the same for 558 metrics, 3 unstable metrics. scenario:iastaspectsospath-ospathsplitdrive_aspect
scenario:iastaspectsospath-ospathsplitext_aspect
scenario:iastaspectssplit-splitlines_aspect
|
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
… add defensive instrumentation try/catch
| span.finish() | ||
| raise |
There was a problem hiding this comment.
| span.finish() | |
| raise | |
| raise | |
| finally: | |
| span.finish() |
?
There was a problem hiding this comment.
We only want to finish the span if there's an error, since we're always expecting a streamed response otherwise (which we'll finish in the TracedStreamResponse wrapper class)
| try: | ||
| result = original_func(*args, **kwargs) | ||
| result = handle_bedrock_agent_response(result, integration, span, args, kwargs) | ||
| return result | ||
| except Exception: | ||
| integration.llmobs_set_tags(span, args, kwargs, result, operation="agent") | ||
| span.set_exc_info(*sys.exc_info()) | ||
| span.finish() | ||
| raise |
There was a problem hiding this comment.
| try: | |
| result = original_func(*args, **kwargs) | |
| result = handle_bedrock_agent_response(result, integration, span, args, kwargs) | |
| return result | |
| except Exception: | |
| integration.llmobs_set_tags(span, args, kwargs, result, operation="agent") | |
| span.set_exc_info(*sys.exc_info()) | |
| span.finish() | |
| raise | |
| try: | |
| result = original_func(*args, **kwargs) | |
| result = handle_bedrock_agent_response(result, integration, span, args, kwargs) | |
| return result | |
| finally: | |
| integration.llmobs_set_tags(span, args, kwargs, result, operation="agent") |
?
There was a problem hiding this comment.
Unfortunately here we only want to call the finish/llmobs_set_tags methods in case of error, otherwise we handle them once the wrapped StreamResponse object consumes all chunks. This is just an error case handling here, I'll add a comment to clarify this
| @@ -1377,7 +1377,7 @@ def select_pys(min_version: str = MIN_PYTHON_VERSION, max_version: str = MAX_PYT | |||
| ), | |||
| Venv( | |||
| pys=select_pys(min_version="3.9"), | |||
| pkgs={"vcrpy": "==7.0.0", "botocore": ">=1.34.131", "boto3": ">=1.34.131"}, | |||
| pkgs={"vcrpy": "==7.0.0", "botocore": "==1.38.26", "boto3": "==1.38.26"}, | |||
There was a problem hiding this comment.
note that we might have to change the min compatible version here: https://github.com/DataDog/dd-trace-py/pull/13613/files#diff-1f8812e8f134ceee7717f2f35d62d2ad3c455933bac5899e932e96c39e5cdf80 - I think @wconti27 was doing something with enforcing minimum version for SSI?
There was a problem hiding this comment.
Do we? I thought the lines above there's a base riot Venv that tests an even older version of botocore 1.34.49
MLOB-2660
This PR adds tracing for AWS Bedrock agents runtime.
There are 2 components to this PR:
invokeAgent()call, i.e. the end-to-end agent request.enableTrace=Truein the request params), then we translate the returned bedrock traces into LLMObs span events.Translating AWS Bedrock Traces to Datadog LLMObs Spans
The main idea here is that we are receiving AWS bedrock "trace" events and converting/stitching them into Datadog LLMObs spans. We do three levels of tracing:
invokeAgentcall.Trace steps and inner trace Steps
As of the time of writing, Bedrock Agent Traces can consist of the following trace types, each with their own internal structure and unique types:
Inner trace step events also include but are not limited to
modelInvocationInput, modelInvocationOutput, actionGroupInvocationInput, actionGroupInvocationOutput. These events need to be stitched together to create LLMObs span events. I've also maderationaleandguardrailtrace step events standalone spans. Also, since we're translating events to LLMObs span dictionaries, things like exceptions specified in the trace step events need to be raised explicitly in our span events in order to retain the traceback information to propagate to the root agent span.For more information on the Bedrock Agent Trace structure/formats, see their docs.
Checklist
Reviewer Checklist