Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks test #56

Merged
merged 10 commits into from
Jan 20, 2024
Merged

Add benchmarks test #56

merged 10 commits into from
Jan 20, 2024

Conversation

immortal-tofu
Copy link
Contributor

No description provided.

Copy link
Collaborator

@dartdart26 dartdart26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

fhevm/benchmarks_test.go Outdated Show resolved Hide resolved
Copy link
Member

@youben11 youben11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking deeper into the functions being benchmarked, aren't they executing much more than just the fhe execution? Perhaps we should see lower level functions to benchmark

@immortal-tofu
Copy link
Contributor Author

Looking deeper into the functions being benchmarked, aren't they executing much more than just the fhe execution? Perhaps we should see lower level functions to benchmark

I don't think so, because otherwise we could simply use tfhe-rs benchmarks. For example, right now we realise there is an issue so it's useful ;)

@youben11
Copy link
Member

Looking deeper into the functions being benchmarked, aren't they executing much more than just the fhe execution? Perhaps we should see lower level functions to benchmark

I don't think so, because otherwise we could simply use tfhe-rs benchmarks. For example, right now we realise there is an issue so it's useful ;)

I would agree if the functions we were benchmarking were fhevm function, but what's currently being benchmarked are test functions that never runs when executing the fhevm, so the numbers can be misleading due to an error in the test code, or an update to the test functions.

@immortal-tofu
Copy link
Contributor Author

The ultimate goal is just to have a rough idea of timings for estimate gas prices. I think it's useful to have these benchmarks. I can close the PR since I get the the values for update gas prices, but I'm pretty sure it will be useful in 3 or 6 months to reestimate gas prices.

@youben11
Copy link
Member

The ultimate goal is just to have a rough idea of timings for estimate gas prices. I think it's useful to have these benchmarks. I can close the PR since I get the the values for update gas prices, but I'm pretty sure it will be useful in 3 or 6 months to reestimate gas prices.

My only fear is that it's confused with fhevm benchmarks while it's a little bit more than that

@immortal-tofu
Copy link
Contributor Author

Would you prefer I rename it to "gas_estimation" ?

@youben11
Copy link
Member

Yes, I guess we can merge it if it's "gas estimation" in the tests and Makefile

Copy link
Member

@youben11 youben11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you forgot to rename the file as well

@immortal-tofu immortal-tofu merged commit 543b143 into main Jan 20, 2024
1 check passed
@immortal-tofu immortal-tofu deleted the feat/benchmarks branch January 20, 2024 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants