Tracing in ML-Commons #4084
Draft
Tracing in ML-Commons #4084
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* adding agent tracing to mlplugin Signed-off-by: chrislai <chrlaii@amazon.com> * add settings and clean code Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * cr fixes Signed-off-by: chrislai <chrlaii@amazon.com> * tests message fix and spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * fix comments Signed-off-by: chrislai <chrlaii@amazon.com> * add visiblefortesting Signed-off-by: chrislai <chrlaii@amazon.com> * address comments Signed-off-by: chrislai <chrlaii@amazon.com> * javadoc Signed-off-by: chrislai <chrlaii@amazon.com> * more tests Signed-off-by: chrislai <chrlaii@amazon.com> * more class fixes Signed-off-by: chrislai <chrlaii@amazon.com> * overloaded method Signed-off-by: chrislai <chrlaii@amazon.com> --------- Signed-off-by: chrislai <chrlaii@amazon.com>
* adding agent tracing to mlplugin Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * cr fixes Signed-off-by: chrislai <chrlaii@amazon.com> * spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * P-E-R tracing Signed-off-by: chrislai <chrlaii@amazon.com> * add input/output to all spans Signed-off-by: chrislai <chrlaii@amazon.com> * per agent runner updates Signed-off-by: chrislai <chrlaii@amazon.com> * remove conversational agent changes Signed-off-by: chrislai <chrlaii@amazon.com> * rebase to main Signed-off-by: chrislai <chrlaii@amazon.com> * propagate tokens Signed-off-by: chrislai <chrlaii@amazon.com> * remove extractJsonBlock Signed-off-by: chrislai <chrlaii@amazon.com> * reconfigure classes and write coverage tests Signed-off-by: chrislai <chrlaii@amazon.com> * reconfig + tests Signed-off-by: chrislai <chrlaii@amazon.com> * address comments Signed-off-by: chrislai <chrlaii@amazon.com> * javadoc and avoid import* Signed-off-by: chrislai <chrlaii@amazon.com> * remove index mapping Signed-off-by: chrislai <chrlaii@amazon.com> * large refactor + testing Signed-off-by: chrislai <chrlaii@amazon.com> * avoid default null Signed-off-by: chrislai <chrlaii@amazon.com> * less null checks Signed-off-by: chrislai <chrlaii@amazon.com> --------- Signed-off-by: chrislai <chrlaii@amazon.com>
* adding agent tracing to mlplugin Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * cr fixes Signed-off-by: chrislai <chrlaii@amazon.com> * spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * P-E-R tracing Signed-off-by: chrislai <chrlaii@amazon.com> * add input/output to all spans Signed-off-by: chrislai <chrlaii@amazon.com> * conversational agent tracing Signed-off-by: chrislai <chrlaii@amazon.com> * allow flexability in tool return format Signed-off-by: chrislai <chrlaii@amazon.com> * directly access singleton + good practice coding Signed-off-by: chrislai <chrlaii@amazon.com> * remove null checks Signed-off-by: chrislai <chrlaii@amazon.com> * rebase off main Signed-off-by: chrislai <chrlaii@amazon.com> * spacing Signed-off-by: chrislai <chrlaii@amazon.com> * propagate tokens Signed-off-by: chrislai <chrlaii@amazon.com> * refactor and add tests Signed-off-by: chrislai <chrlaii@amazon.com> * fix rebase issues Signed-off-by: chrislai <chrlaii@amazon.com> * huge refactor Signed-off-by: chrislai <chrlaii@amazon.com> * clean code Signed-off-by: chrislai <chrlaii@amazon.com> * spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * more refactoring Signed-off-by: chrislai <chrlaii@amazon.com> * more helper functions Signed-off-by: chrislai <chrlaii@amazon.com> * reformat listener Signed-off-by: chrislai <chrlaii@amazon.com> * major refactor + address comments Signed-off-by: chrislai <chrlaii@amazon.com> --------- Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
…pensearch-project#3976) * adding agent tracing to mlplugin Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * cr fixes Signed-off-by: chrislai <chrlaii@amazon.com> * spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * P-E-R tracing Signed-off-by: chrislai <chrlaii@amazon.com> * add input/output to all spans Signed-off-by: chrislai <chrlaii@amazon.com> * conversational agent tracing Signed-off-by: chrislai <chrlaii@amazon.com> * allow flexability in tool return format Signed-off-by: chrislai <chrlaii@amazon.com> * directly access singleton + good practice coding Signed-off-by: chrislai <chrlaii@amazon.com> * convflow and flow agent tracing Signed-off-by: chrislai <chrlaii@amazon.com> * new settings and noop update Signed-off-by: chrislai <chrlaii@amazon.com> * overloading method Signed-off-by: chrislai <chrlaii@amazon.com> * propagate tokens Signed-off-by: chrislai <chrlaii@amazon.com> * rebase cleaning Signed-off-by: chrislai <chrlaii@amazon.com> * clean Signed-off-by: chrislai <chrlaii@amazon.com> * clean Signed-off-by: chrislai <chrlaii@amazon.com> * javadoc Signed-off-by: chrislai <chrlaii@amazon.com> * rebase to main Signed-off-by: chrislai <chrlaii@amazon.com> * refactoring code Signed-off-by: chrislai <chrlaii@amazon.com> * refactoring Signed-off-by: chrislai <chrlaii@amazon.com> * address comments Signed-off-by: chrislai <chrlaii@amazon.com> * fixed numbering Signed-off-by: chrislai <chrlaii@amazon.com> * address comments Signed-off-by: chrislai <chrlaii@amazon.com> --------- Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
* adding agent tracing to mlplugin Signed-off-by: chrislai <chrlaii@amazon.com> * add settings and clean code Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * cr fixes Signed-off-by: chrislai <chrlaii@amazon.com> * tests message fix and spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * spotlessApply Signed-off-by: chrislai <chrlaii@amazon.com> * add tests Signed-off-by: chrislai <chrlaii@amazon.com> * fix comments Signed-off-by: chrislai <chrlaii@amazon.com> * add visiblefortesting Signed-off-by: chrislai <chrlaii@amazon.com> * address comments Signed-off-by: chrislai <chrlaii@amazon.com> * javadoc Signed-off-by: chrislai <chrlaii@amazon.com> * more tests Signed-off-by: chrislai <chrlaii@amazon.com> * more class fixes Signed-off-by: chrislai <chrlaii@amazon.com> * overloaded method Signed-off-by: chrislai <chrlaii@amazon.com> --------- Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Signed-off-by: chrislai <chrlaii@amazon.com>
Agent Tracing Storage and Visualization Tutorial
chriswlai
had a problem deploying
to
ml-commons-cicd-env-require-approval
GitHub Actions
Failure
chriswlai
had a problem deploying
to
ml-commons-cicd-env-require-approval
GitHub Actions
Failure
chriswlai
had a problem deploying
to
ml-commons-cicd-env-require-approval
GitHub Actions
Failure
chriswlai
had a problem deploying
to
ml-commons-cicd-env-require-approval
GitHub Actions
Failure
5 tasks
pyek-bot
reviewed
Sep 2, 2025
Comment on lines
+477
to
+479
| long llmLatency = System.currentTimeMillis() - llmStartTime; | ||
|
|
||
| Map<String, Integer> extractedTokens = MLAgentTracer.extractTokensFromModelOutput(modelTensorOutput, allParams); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This feature adds tracing to ML-Commons. More details can be found here: https://quip-amazon.com/Nc4RA8w6XIwa/Design-Doc-ML-Commons-Agent-Tracing-Observability
Related Issues
Resolves #3971
Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.