WIP: rebasing @localbtime and friends from LocalScopeBenchmarks.jl#150
Draft
miguelraz wants to merge 5 commits intoJuliaCI:mainfrom
Draft
WIP: rebasing @localbtime and friends from LocalScopeBenchmarks.jl#150miguelraz wants to merge 5 commits intoJuliaCI:mainfrom
miguelraz wants to merge 5 commits intoJuliaCI:mainfrom
Conversation
Author
|
The tests are failing / and passing due to a brittle judging function. One idea would be to judge loosely min, mean and median and if 2/3 pass then the judgement is considered invariant. |
Collaborator
|
Hey @miguelraz, just got |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a WIP PR to merge a
@localbtimeand friends macro into BenchmarkTools.jl .There's a thread with reported issues here and this PR took existing code from
LocalScopeBenchmarks.jl, with all credit to @rdeits.This PR:
@localbenchmark,@localbtime,@localbelapsed1All feedback is welcome.
Another welcome feedback will be on how the new tests for local time regression measurement should be done: I found that when running them locally, the first run would pass all tests, then fail due only to the regression threshold set. I think it's hitting the microbenchmarks remembering cache history.
Lastly, this PR does not:
@btimeet al.That is left as a discussion for another time, and for now I will focus on landing this functionality.
Ping also to @jlperla, @ChrisRackauckas.