Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: session end analytics: tools, llms, duration #392

Merged
merged 4 commits into from
Sep 18, 2024

Conversation

heethjain21
Copy link
Contributor

📥 Pull Request

📘 Description

  • Counting the no of events based on event type
  • Logging the no of events on session end
  • Also logging the session duration

🔄 Related Issue (if applicable)
Add duration + LLM analytics at end of session in CLI #380

🎯 Goal

  • Have some additional metrics shown on session ending

🔍 Additional Context

  • I checked if there was an API for GET session by Id, but couldn't find it, so the next best solution was to just simply calculate the events

🧪 Testing

  • Not needed, since its just logging

@HowieG
Copy link
Contributor

HowieG commented Sep 17, 2024

Hey @heethjain21 - we're able to calcuate these API-side and for duration we do have that in our dashboard as one of our key features - https://app.agentops.ai/
So we don't have a need for this PR unfortunately

@HowieG
Copy link
Contributor

HowieG commented Sep 17, 2024

I guess the intention was to log them at the end just like we log the cost. Again not sure if there's a need and I think we have to keep the terminal more or less clean because developers that use our sdk may not want a lot of noise especially considering we're an add-on for observability not like a primary tool for accomplishing a task

@HowieG
Copy link
Contributor

HowieG commented Sep 17, 2024

Actually I think duration is valuable! Asking team about counts

@HowieG HowieG merged commit ffa5a15 into AgentOps-AI:main Sep 18, 2024
3 of 10 checks passed
@heethjain21
Copy link
Contributor Author

Hi @HowieG ,

This PR's workflow didn't run all the tests (earlier), so couldn't realize dateutil package messed up.
I had it installed the package locally, so couldn't replicate it for the tests locally (and hence couldn't catch the tests failure)

I have corrected the issue in this Pull Request: #387
Please take a look, and merge it so the PR workflows get fixed.

@areibman
Copy link
Contributor

Thanks @heethjain21 !

@HowieG
Copy link
Contributor

HowieG commented Sep 18, 2024

Hey @heethjain21 thanks for followup my bad I also merged without checking the workflows. I ended up replacing the dateutil logic with the builtin datetime to minimize addition of new dependencies

@heethjain21
Copy link
Contributor Author

Alright, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants