-
Notifications
You must be signed in to change notification settings - Fork 134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make benchmarks more precises and complete? #440
Comments
It should be quite obvious that these benchmarks are to make certain languages look faster than others. The number of PR's and Issues untouched for 3 years proves that point. I was going to fix some code that I saw as blatantly slow but its clear to me that this is a marketing campaign vs a real desire to get the best out of each language. |
@rmannibucau Do you realize that the workload is expected to be set on the command-line as a program arg? @hanabi1224 set N=18 on the command-line for binary-trees and the benchmarks game set N=21. |
@igouy doesnt change critically the output (didnt test monstereous number neither) but still think points are valid |
You're reporting tiny tiny durations, 1000ths of a second, when the resolution of the time command may only be 1000 Hz.
java # 7 is insanely fast. The other java and c# binary trees programs are similar too each other. |
Ok, seems it was updated since I ran it cause it was not these numbers at all (or I misused something). These numbers are ok for me. |
Hi,
I was interested by java vs csharp comparison.
I was quite surprised by the exposed results - in particular the bintree one where dotnet looks insanely slow - so I tested it locally.
I don't get exactly the same results (basically I fall into the error zone).
Here is what I did:
time
Here are my results (did multiple runs and it moves from ~20% since the execution is very fast):
What's important to note is that we can't conclude dotnet is faster than java in native mode, if you run 100 times dotnet will statically be slower but can be overall faster - this is why I think the soft is too short and dotnet has so much rate adjustment than without tuning for such a short live execution you get this instability.
Note: i did all the bench on ubuntu with 16 i9 and 64G of ram (indeed way too much for these apps ;)) and no particular tuning.
What is important for me is:
experience)
That said I still want to also say a big thank you cause it is a lot of work and always a very source to get started when working on these topics.
The text was updated successfully, but these errors were encountered: