Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get more data from benchmark statement #329

Open
lesquoyb opened this issue Sep 17, 2024 · 1 comment
Open

Get more data from benchmark statement #329

lesquoyb opened this issue Sep 17, 2024 · 1 comment
Labels
About GAML This issue concerns the GAML language 🤗 Enhancement This is a request for enhancement

Comments

@lesquoyb
Copy link
Contributor

Is your request related to a problem? Please describe.
Currently the benchmark statement is pretty useful to compare things "manually" but it is not meant to be handled programmatically.
For example if I want to study the impact of one variable on the average execution time using this statement, I will have to copy the content of the console into an external text editor and do a lot of string manipulation to transform it into something usable like a csv to display it as a chart in yet another program. Which is a bit sad because we have all the data available in gama and the tools to display it (or export to csv if I need more serious external tools) but everything ends up in the console.

Describe the improvement you'd like
My proposal is that we add a facet to the benchmark statement to store the duration of each repetition into a list, and maybe another one to turn on or off the writing into the console. This way we could let the user manipulate the results however they want and possibly calculate other metrics they are interested into like standard deviation.

Describe alternatives you've considered
Switch from a statement into an operator that will either return the string we currently have, or a list of duration, depending on the options we set. That would be preferable in my opinion but that would also break previous models using it and probably will take more time implementing

@lesquoyb lesquoyb added 🤗 Enhancement This is a request for enhancement About GAML This issue concerns the GAML language labels Sep 17, 2024
@AlexisDrogoul
Copy link
Member

Sorry to have not reacted before but the benchmark option in experiments gives you access to raw data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
About GAML This issue concerns the GAML language 🤗 Enhancement This is a request for enhancement
Projects
None yet
Development

No branches or pull requests

2 participants