-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add progress bar to allocation preparation #53
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems legit :) Not sure the idiomatic way to do this in Julia, but I think this could be sped up by making this loop into a "parallel map" — i.e. decoding the allocs in parallel on multiple cores. But that should be a separate PR…
Codecov Report
@@ Coverage Diff @@
## master #53 +/- ##
=======================================
Coverage 72.81% 72.81%
=======================================
Files 3 3
Lines 298 298
=======================================
Hits 217 217
Misses 81 81
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😊 thanks Ian, that's great. 👍 LGTM
Reading this again, I realized that a significant amount of time is spent in I.e. out of two stages — decode and pprof — this is only capturing pprof. Kind of makes me want to bring back JuliaLang/julia#43889, which deferred stack decoding so basically all the time would be in pprof 🤔 |
Yeah, @vilterp: I think that we should provide another API that only exports the raw data together with something like an We talked about something like that in the past, i think. |
Note that |
Yes, I think we should be able to do something similar :) Also, for what it's worth, the strategy there of partitioning a collection and spawning a thread to process each partition seems like a good addition to the standard library (i.e. |
Exactly.. JuliaLang/julia#43919 |
I ran the allocation profiler with
sample_rate=0.1
and the process of preparing the pprof took >10 minutes with no output before I quit.This adds a progress meter to the preparation stage.
There's still a long silence before the warning message shows, but I guess that's a
Profile
issue.cc. @NHDaly @vilterp