Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[optimization] add a way to log each function call and duration #342

Closed
lesquoyb opened this issue Sep 26, 2024 · 3 comments
Closed

[optimization] add a way to log each function call and duration #342

lesquoyb opened this issue Sep 26, 2024 · 3 comments
Labels
🤗 Enhancement This is a request for enhancement

Comments

@lesquoyb
Copy link
Contributor

Describe the bug
While writing documentation about how to optimize models I noticed that in some case you have no choice but to log every action/reflex called and their duration to get an idea of where to look for optimizations.
This process is quite tedious and always the same so it's perfect for automating and I was thinking that we could provide to the users an execution mode that would automatically do it and export the results in a csv or something to be visualized in an external program

@lesquoyb lesquoyb added the 🤗 Enhancement This is a request for enhancement label Sep 26, 2024
@AlexisDrogoul
Copy link
Member

Isnt'it the purpose of the existing benchmark mode ?

@lesquoyb
Copy link
Contributor Author

Benchmark only works for the code contained inside of it, I was thinking about an execution mode that would basically wrap every action/reflex by an equivalent of benchmark.
Another problem with benchmark is that it doesn't produce raw data and do not give access to it in other way than a text in the console (as mentioned in #329)

@lesquoyb
Copy link
Contributor Author

So to give some updates on that, there's a benchmark mode on the experiment level, which is different from the statement I was thinking about.
This mode records all lines of code executed with the number of calls and accumulated duration.
I've added this information to the page I'm currently building on optimization so closing this issue, but I'm quite that if people actually use it, it will need to be refined a bit

@lesquoyb lesquoyb closed this as not planned Won't fix, can't repro, duplicate, stale Sep 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤗 Enhancement This is a request for enhancement
Projects
None yet
Development

No branches or pull requests

2 participants