Skip to content

Conversation

@luke-moehlenbrock
Copy link
Contributor

@luke-moehlenbrock luke-moehlenbrock commented Oct 22, 2025

Adds support for routingClassifierTraces and also splits out the guardrails to pre and post guardrails as separate spans.


Note

Adds routing classifier trace support, splits guardrails into pre/post spans, improves metadata propagation and parent attr extraction, updates trace ID handling, and extends tests.

  • Instrumentation (Bedrock):
    • Routing Classifier: Add routingClassifierTrace handling across collector and accumulator; parse outputs from rawResponse.content and set CHAIN span with input/output and metadata.
    • Guardrails: Split into preGuardrailTrace/postGuardrailTrace spans; name Guardrails spans accordingly and compute unique IDs differently; propagate guardrail metadata and set span status based on interventions.
    • Metadata propagation: Merge invocation/observation metadata into spans (incl. agent-collaborator); prefer trace-level metadata; set parent span input/output from node-level chunks; include routing/orchestration/pre/post metadata on spans.
    • Agent collaborator: Merge provided metadata with agent fields for invocation input/output; name spans invocationType[agentName].
    • Trace collection: Add node-level chunk storage for non-invocation chunks; attach chunks on node creation; adjust unique trace ID generation for guardrails.
  • Tests:
    • Add routing classifier cassette and test; expand tests to assert orchestration/pre/post metadata and updated guardrail behavior; adjust expected span counts/names.

Written by Cursor Bugbot for commit 70340c2. This will update automatically on new commits. Configure here.

@luke-moehlenbrock luke-moehlenbrock requested a review from a team as a code owner October 22, 2025 23:33
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 22, 2025
cursor[bot]

This comment was marked as outdated.

# or if content was not valid JSON
return attributes.request_attributes.update(
get_output_attributes(raw_response_content)
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luke-moehlenbrock I think this is fine given what get_output_attributes does, but I think it's valid feedback that raw_response_content should ideally not be re-assigned, so that it's easier to follow and so the the data takes on something other than what it's name is describing. I'd rename 390 to something else and then use that in a return statement within the try block if the subsequent isn't true (i.e. no 'output" on the json)

return attributes.request_attributes.update(
get_output_attributes(output_text)
)
# For Routing classifier events, the output is in rawResponse.content
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does input data/ metadata data show up correctly for routing classifier traces?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call on raw_response_content, I'll change the name to something a little more clear.

As for the routingClassifierTrace, it does have the input/output data from the LLM call, but the metadata isn't captured. This is due to how the TraceNodes are handled, the function loops through the chunks of the trace_span_data and uses the events in the chunks to populate the span attributes, but it also sets the name of the span. If we add the trace data as a chunk to the TraceNode then _prepare_span_attributes will see the modelInvocationInput or modelInvocationOutput attributes in the trace data which will cause it to reassign the name to LLM instead of routingClassifierTrace or orchestrationTrace. We could probably get around that issue though by adding a _process_trace_node method that only extracts the metadata from the chunks without reassigning the span name

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Oct 23, 2025
"""
if chunk_type not in ["invocationInput", "modelInvocationInput"]:
# Add chunk to trace node as well, useful for propogating metadata to the parent node
node.add_chunk(trace_data)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Trace Data Duplication Causes Processing Errors

Potential duplicate chunk data: The code adds trace_data to node.chunks in _handle_chunk_for_current_node (line 219), but trace_data is also added to node.chunks when creating new trace nodes in _handle_new_trace_node (lines 257 and 263). This means the same trace_data could be added twice to the TraceNode's chunks list - once during node creation and once during chunk processing. This duplication could lead to incorrect metadata extraction or span attribute processing as the same data would be processed multiple times when iterating over node.chunks in _process_trace_node.

Fix in Cursor Fix in Web

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, the first chunk will have invocationInput or modelInvocationInput, this might not always be the case but it should be the case for most TraceNodes. In the case where we add a duplicate chunk to a node, it shouldn't have any impact. The function that processes span attributes loops through the chunks in a node and uses each chunk to write attributes to the _attributes object. For duplicate chunks, the second chunk would just overwrite the attributes with the exact same values so there shouldn't be any unexpected behaviour here. We need to add the chunks here to ensure that routingClassifierTrace and orchestrationTrace properly capture the output from their respective events.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants