[EPIC] Observability improvements

[EItanya]

One of the most important part of any agentic system is the ability to introspect and understand the system's actions, and even intentions. There are some aspects of agentic observability that are incredibly unique, but much of it is built from the same building blocks as any other observability solution.

The library we are built on top of, Autogen, has very powerful logging/tracing solutions, but there are certain things we can change as well as add on top to deliver additional value.

1. Metrics:

• Add metrics for total LLM calls/token-count
• Add metrics for total tool calls
• Add metrics for total Agent calls
• Add the ability to customize labels based on metadata of agent/tool requests

2. Logging:

• Add more explicit oTEL integrations

3. Tracing

• Potential integration with a project like openinference
• Add instrumentation for all LLM providers, right now it's only OpenAI

[nimishamehta5]

+1 on this issue, will bring Kagent closer to a production-grade application

Has any work been done on this front yet @EItanya? Would love to discuss how to get this started if not.

[EItanya]

1000%. I spoke to @dimetron about this the other day as well, and I know it's important to nearly everyone. We have not been able to pick this up internally yet, but we would love to work with the community to get this done as it's absolutely key to kagent's success

[braghettos]

This is a super-useful feature in use cases like chargeback of model usage per user!

[github-actions]

This issue has been marked as stale because of no activity in the last 90 days. It will be closed in the next 90 days unless it is tagged "no stalebot" or other activity occurs.