-
Couldn't load subscription status.
- Fork 148
feat(bedrock): Add Routing Classifier Trace event #2342
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| # or if content was not valid JSON | ||
| return attributes.request_attributes.update( | ||
| get_output_attributes(raw_response_content) | ||
| ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@luke-moehlenbrock I think this is fine given what get_output_attributes does, but I think it's valid feedback that raw_response_content should ideally not be re-assigned, so that it's easier to follow and so the the data takes on something other than what it's name is describing. I'd rename 390 to something else and then use that in a return statement within the try block if the subsequent isn't true (i.e. no 'output" on the json)
| return attributes.request_attributes.update( | ||
| get_output_attributes(output_text) | ||
| ) | ||
| # For Routing classifier events, the output is in rawResponse.content |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does input data/ metadata data show up correctly for routing classifier traces?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call on raw_response_content, I'll change the name to something a little more clear.
As for the routingClassifierTrace, it does have the input/output data from the LLM call, but the metadata isn't captured. This is due to how the TraceNodes are handled, the function loops through the chunks of the trace_span_data and uses the events in the chunks to populate the span attributes, but it also sets the name of the span. If we add the trace data as a chunk to the TraceNode then _prepare_span_attributes will see the modelInvocationInput or modelInvocationOutput attributes in the trace data which will cause it to reassign the name to LLM instead of routingClassifierTrace or orchestrationTrace. We could probably get around that issue though by adding a _process_trace_node method that only extracts the metadata from the chunks without reassigning the span name
| """ | ||
| if chunk_type not in ["invocationInput", "modelInvocationInput"]: | ||
| # Add chunk to trace node as well, useful for propogating metadata to the parent node | ||
| node.add_chunk(trace_data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Trace Data Duplication Causes Processing Errors
Potential duplicate chunk data: The code adds trace_data to node.chunks in _handle_chunk_for_current_node (line 219), but trace_data is also added to node.chunks when creating new trace nodes in _handle_new_trace_node (lines 257 and 263). This means the same trace_data could be added twice to the TraceNode's chunks list - once during node creation and once during chunk processing. This duplication could lead to incorrect metadata extraction or span attribute processing as the same data would be processed multiple times when iterating over node.chunks in _process_trace_node.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally, the first chunk will have invocationInput or modelInvocationInput, this might not always be the case but it should be the case for most TraceNodes. In the case where we add a duplicate chunk to a node, it shouldn't have any impact. The function that processes span attributes loops through the chunks in a node and uses each chunk to write attributes to the _attributes object. For duplicate chunks, the second chunk would just overwrite the attributes with the exact same values so there shouldn't be any unexpected behaviour here. We need to add the chunks here to ensure that routingClassifierTrace and orchestrationTrace properly capture the output from their respective events.
Adds support for routingClassifierTraces and also splits out the guardrails to pre and post guardrails as separate spans.
Note
Adds routing classifier trace support, splits guardrails into pre/post spans, improves metadata propagation and parent attr extraction, updates trace ID handling, and extends tests.
routingClassifierTracehandling across collector and accumulator; parse outputs fromrawResponse.contentand set CHAIN span with input/output and metadata.preGuardrailTrace/postGuardrailTracespans; name Guardrails spans accordingly and compute unique IDs differently; propagate guardrail metadata and set span status based on interventions.metadatawith agent fields for invocation input/output; name spansinvocationType[agentName].Written by Cursor Bugbot for commit 70340c2. This will update automatically on new commits. Configure here.