-
Notifications
You must be signed in to change notification settings - Fork 0
fix(testing): incomplete reading of benchmark time in b.N loops #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(testing): incomplete reading of benchmark time in b.N loops #43
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes a critical bug in the benchmark timing infrastructure where SaveMeasurement() was being called before the timer was stopped, leading to incomplete or inaccurate duration measurements in b.N loop benchmarks. The fix adds a runtime panic guard to detect this condition and reorders the method calls to ensure measurements are only saved after timers have been properly stopped.
Key changes:
- Added a panic guard in
SaveMeasurement()to prevent it from being called whiletimerOnis true - Reordered
SaveMeasurement()to be called afterStopTimer()in__codspeed_root_frame__runN() - Reordered
SaveMeasurement()to be called afterStopTimerWithoutMarker()inLoop()
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| testing/testing/benchmark.go | Added panic guard in SaveMeasurement() and reordered timer stop/measurement save calls in runN() and Loop() to ensure timer is stopped before reading duration |
| testing/patches/benchmark_savemeasurement_bug.patch | Patch file containing the same fixes to be applied during the fork process to Go's standard testing package |
| testing/fork.sh | Added application of the new benchmark_savemeasurement_bug.patch to the build process |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
CodSpeed Performance ReportMerging #43 will degrade performances by 29.54%Comparing Summary
Benchmarks breakdown
|
adriencaccia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Is such a speedup intended? https://codspeed.io/CodSpeedHQ/codspeed-go/branches/cod-1776-go-benchmarks-report-1ns-time-for-fuego?utm_source=github&utm_medium=comment&utm_content=header&uri=go-runner%2Fbenches%2Fgo_runner.rs%3A%3Abench_go_runner&runnerMode=WallTime§ionId=benchmark-comparison-section-comparison-improved
It's a flaky benchmark, we're likely bottlenecked by the disk read/write speed since we're copying, patching and building a Go project. |

No description provided.