-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allocation profiler #42768
Allocation profiler #42768
Commits on Jan 7, 2022
-
Configuration menu - View commit details
-
Copy full SHA for b49dd24 - Browse repository at this point
Copy the full SHA b49dd24View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2fe1678 - Browse repository at this point
Copy the full SHA 2fe1678View commit details -
Configuration menu - View commit details
-
Copy full SHA for c370f5a - Browse repository at this point
Copy the full SHA c370f5aView commit details -
Configuration menu - View commit details
-
Copy full SHA for b8cffd3 - Browse repository at this point
Copy the full SHA b8cffd3View commit details -
Configuration menu - View commit details
-
Copy full SHA for c733cd6 - Browse repository at this point
Copy the full SHA c733cd6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 964a8a4 - Browse repository at this point
Copy the full SHA 964a8a4View commit details -
Configuration menu - View commit details
-
Copy full SHA for aed5f5d - Browse repository at this point
Copy the full SHA aed5f5dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 665cacd - Browse repository at this point
Copy the full SHA 665cacdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7ed832e - Browse repository at this point
Copy the full SHA 7ed832eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 371bb96 - Browse repository at this point
Copy the full SHA 371bb96View commit details -
Configuration menu - View commit details
-
Copy full SHA for c8452ce - Browse repository at this point
Copy the full SHA c8452ceView commit details -
not sure it's actually freeing, but at least it doesn't crash
Configuration menu - View commit details
-
Copy full SHA for 2d9ec5b - Browse repository at this point
Copy the full SHA 2d9ec5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5cd096a - Browse repository at this point
Copy the full SHA 5cd096aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 04e2b17 - Browse repository at this point
Copy the full SHA 04e2b17View commit details -
Configuration menu - View commit details
-
Copy full SHA for 77ce036 - Browse repository at this point
Copy the full SHA 77ce036View commit details -
Configuration menu - View commit details
-
Copy full SHA for 33dde11 - Browse repository at this point
Copy the full SHA 33dde11View commit details -
Configuration menu - View commit details
-
Copy full SHA for 07d1009 - Browse repository at this point
Copy the full SHA 07d1009View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ec2fd0 - Browse repository at this point
Copy the full SHA 6ec2fd0View commit details -
send type names over to the Julia side
kind of annoying, but it doesn't seem we can reliably load them from the addresses without segfaulting
Configuration menu - View commit details
-
Copy full SHA for f5f1be2 - Browse repository at this point
Copy the full SHA f5f1be2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9fe1ad2 - Browse repository at this point
Copy the full SHA 9fe1ad2View commit details -
but I think the problem is that the strings are being GC'd cuz they're not rooted
Configuration menu - View commit details
-
Copy full SHA for f768234 - Browse repository at this point
Copy the full SHA f768234View commit details -
malloc strings so they're not junk by the time we look at them
this works, but all the strings leak ugh C is hard
Configuration menu - View commit details
-
Copy full SHA for b11a0fe - Browse repository at this point
Copy the full SHA b11a0feView commit details -
implement caching of stack trace lookups
like the Profile does, and the C++ side did before wonder if we can move this into stacktraces.jl or something to dedup with Profile
Configuration menu - View commit details
-
Copy full SHA for c1faa44 - Browse repository at this point
Copy the full SHA c1faa44View commit details -
add GC.@preserve, magically fixing segfaults (???)
also change Julia representation of raw backtrace elements, thogh I'm not sure it was breaking anything
Configuration menu - View commit details
-
Copy full SHA for 671b88e - Browse repository at this point
Copy the full SHA 671b88eView commit details -
delete code to stringify type names on C++ side
thanks to GC.@preserve <3
Configuration menu - View commit details
-
Copy full SHA for 6932831 - Browse repository at this point
Copy the full SHA 6932831View commit details -
Configuration menu - View commit details
-
Copy full SHA for e164cc8 - Browse repository at this point
Copy the full SHA e164cc8View commit details -
Mark the AllocProfile C code as
JL_NOTSAFEPOINT
TODO: is this correct!?
Configuration menu - View commit details
-
Copy full SHA for c736dbc - Browse repository at this point
Copy the full SHA c736dbcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9407a91 - Browse repository at this point
Copy the full SHA 9407a91View commit details -
Configuration menu - View commit details
-
Copy full SHA for f297d94 - Browse repository at this point
Copy the full SHA f297d94View commit details -
Configuration menu - View commit details
-
Copy full SHA for 50dcb63 - Browse repository at this point
Copy the full SHA 50dcb63View commit details -
Configuration menu - View commit details
-
Copy full SHA for 976d2bc - Browse repository at this point
Copy the full SHA 976d2bcView commit details -
Configuration menu - View commit details
-
Copy full SHA for e1b2221 - Browse repository at this point
Copy the full SHA e1b2221View commit details -
Configuration menu - View commit details
-
Copy full SHA for 03a0102 - Browse repository at this point
Copy the full SHA 03a0102View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7975c77 - Browse repository at this point
Copy the full SHA 7975c77View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2c84bb9 - Browse repository at this point
Copy the full SHA 2c84bb9View commit details -
Configuration menu - View commit details
-
Copy full SHA for b49606d - Browse repository at this point
Copy the full SHA b49606dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 84745bf - Browse repository at this point
Copy the full SHA 84745bfView commit details -
Configuration menu - View commit details
-
Copy full SHA for a03cfe3 - Browse repository at this point
Copy the full SHA a03cfe3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 79427ca - Browse repository at this point
Copy the full SHA 79427caView commit details -
Add precompile statements to AllocProfile package
This _drastically_ speeds up the tests, for reasons I don't exactly understand.. I wonder if it was messing up some heuristics and deciding to interpret the code instead of compiling it, and had some weird corneer cases in the interpreted code or something? I dunno! But anyway, this drastically speeds it up, so 🤷 sounds like not our problem 😊
Configuration menu - View commit details
-
Copy full SHA for 9f63529 - Browse repository at this point
Copy the full SHA 9f63529View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d12971 - Browse repository at this point
Copy the full SHA 7d12971View commit details -
Malloc right-sized buffers for backtraces.
Instead of allocating a maximum-sized buffer for each backtrace, we keep a single max-sized buffer as a scratch space, write the backtrace to it, and then once we know the size, we allocate a right-sized buffer for the backtrace and copy it over. Benchmark results (measured time for profiling allocations on internal Arroyo benchmark, with `skip_every=0`): This only slightly improves the time to record an alloctions profile: - Before: 275.082525 seconds - After: 245.891006 seconds But it drastically improves the memory usage once the profiling is completed, according to System Activity Monitor: - Before: 17.35 GB - After: 6.92 GB - (Compared to 350 MB for the same task without profiling) We could probably slightly improve the time overhead still furthur by using a single big vector instead of a bunch of individual allocated buffers, but this is probably about the best we could do in terms of space usage. This would allow us to eliminate the redundant copying, and would also amortize away the allocations of the buffers, both of which should reduce the performance impact. But I'm guessing the time is mostly dominated by just how long the stack traces are, and there's no getting around that. At best, we could expect maybe like a 2x-3x improvement from those changes, I think.
Configuration menu - View commit details
-
Copy full SHA for a26f375 - Browse repository at this point
Copy the full SHA a26f375View commit details -
Configuration menu - View commit details
-
Copy full SHA for b5ab611 - Browse repository at this point
Copy the full SHA b5ab611View commit details -
Configuration menu - View commit details
-
Copy full SHA for d53afa2 - Browse repository at this point
Copy the full SHA d53afa2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9de01c6 - Browse repository at this point
Copy the full SHA 9de01c6View commit details -
Configuration menu - View commit details
-
Copy full SHA for cdabd0e - Browse repository at this point
Copy the full SHA cdabd0eView commit details -
Configuration menu - View commit details
-
Copy full SHA for ba3c0c8 - Browse repository at this point
Copy the full SHA ba3c0c8View commit details -
Configuration menu - View commit details
-
Copy full SHA for ddf945c - Browse repository at this point
Copy the full SHA ddf945cView commit details -
I think these were added to find why it was slow, which was fixed by the __precompile__, so we don't need them anymore?
Configuration menu - View commit details
-
Copy full SHA for e2a8ba3 - Browse repository at this point
Copy the full SHA e2a8ba3View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad1b0e0 - Browse repository at this point
Copy the full SHA ad1b0e0View commit details -
Configuration menu - View commit details
-
Copy full SHA for a712267 - Browse repository at this point
Copy the full SHA a712267View commit details -
Configuration menu - View commit details
-
Copy full SHA for ec6cc6f - Browse repository at this point
Copy the full SHA ec6cc6fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 83ef9ea - Browse repository at this point
Copy the full SHA 83ef9eaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 44073ef - Browse repository at this point
Copy the full SHA 44073efView commit details -
Configuration menu - View commit details
-
Copy full SHA for b3e5102 - Browse repository at this point
Copy the full SHA b3e5102View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0fd988b - Browse repository at this point
Copy the full SHA 0fd988bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0119ce3 - Browse repository at this point
Copy the full SHA 0119ce3View commit details -
switch to uniform sampling with rand()
instead of recording any N allocs
Configuration menu - View commit details
-
Copy full SHA for 9d06356 - Browse repository at this point
Copy the full SHA 9d06356View commit details -
Co-authored-by: Nathan Daly <NHDaly@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f0d7da4 - Browse repository at this point
Copy the full SHA f0d7da4View commit details -
Configuration menu - View commit details
-
Copy full SHA for cecf3a1 - Browse repository at this point
Copy the full SHA cecf3a1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 545d06c - Browse repository at this point
Copy the full SHA 545d06cView commit details -
Configuration menu - View commit details
-
Copy full SHA for e1f8684 - Browse repository at this point
Copy the full SHA e1f8684View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7de1d5f - Browse repository at this point
Copy the full SHA 7de1d5fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7c862da - Browse repository at this point
Copy the full SHA 7c862daView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2c271ab - Browse repository at this point
Copy the full SHA 2c271abView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cb0a59 - Browse repository at this point
Copy the full SHA 2cb0a59View commit details -
Configuration menu - View commit details
-
Copy full SHA for 535c2fc - Browse repository at this point
Copy the full SHA 535c2fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for af8931c - Browse repository at this point
Copy the full SHA af8931cView commit details -
Configuration menu - View commit details
-
Copy full SHA for f47253c - Browse repository at this point
Copy the full SHA f47253cView commit details -
the big dict is not worth the overhead for now
Configuration menu - View commit details
-
Copy full SHA for 423830d - Browse repository at this point
Copy the full SHA 423830dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4a86866 - Browse repository at this point
Copy the full SHA 4a86866View commit details -
Configuration menu - View commit details
-
Copy full SHA for d02a5b6 - Browse repository at this point
Copy the full SHA d02a5b6View commit details -
Configuration menu - View commit details
-
Copy full SHA for bfd6210 - Browse repository at this point
Copy the full SHA bfd6210View commit details -
Configuration menu - View commit details
-
Copy full SHA for ff00317 - Browse repository at this point
Copy the full SHA ff00317View commit details -
Configuration menu - View commit details
-
Copy full SHA for e07e67d - Browse repository at this point
Copy the full SHA e07e67dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ed4cb44 - Browse repository at this point
Copy the full SHA ed4cb44View commit details -
Configuration menu - View commit details
-
Copy full SHA for c27971b - Browse repository at this point
Copy the full SHA c27971bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 48261b5 - Browse repository at this point
Copy the full SHA 48261b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 43276f4 - Browse repository at this point
Copy the full SHA 43276f4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8062b72 - Browse repository at this point
Copy the full SHA 8062b72View commit details -
Configuration menu - View commit details
-
Copy full SHA for aeec3b6 - Browse repository at this point
Copy the full SHA aeec3b6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 00bb947 - Browse repository at this point
Copy the full SHA 00bb947View commit details -
Update stdlib/Profile/src/Allocs.jl
Capitalization.
Configuration menu - View commit details
-
Copy full SHA for d3d868f - Browse repository at this point
Copy the full SHA d3d868fView commit details -
Configuration menu - View commit details
-
Copy full SHA for d352a80 - Browse repository at this point
Copy the full SHA d352a80View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3aaa9d9 - Browse repository at this point
Copy the full SHA 3aaa9d9View commit details -
Configuration menu - View commit details
-
Copy full SHA for db64f63 - Browse repository at this point
Copy the full SHA db64f63View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5c25bcd - Browse repository at this point
Copy the full SHA 5c25bcdView commit details -
Apply suggestions from code review
Fixup some small typos and formatting changes
Configuration menu - View commit details
-
Copy full SHA for 4fec8dd - Browse repository at this point
Copy the full SHA 4fec8ddView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9a4b7fb - Browse repository at this point
Copy the full SHA 9a4b7fbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5dc2beb - Browse repository at this point
Copy the full SHA 5dc2bebView commit details -
Merge pull request #13 from vilterp/pv-alloc-profile-docs
alloc profiler: add docs and news
Configuration menu - View commit details
-
Copy full SHA for 830e2ad - Browse repository at this point
Copy the full SHA 830e2adView commit details
Commits on Jan 11, 2022
-
Configuration menu - View commit details
-
Copy full SHA for cd14357 - Browse repository at this point
Copy the full SHA cd14357View commit details -
Merge pull request #16 from vilterp/pv-alloc-profile-style-renames
alloc profiler: rename structs to match Julia C style
Configuration menu - View commit details
-
Copy full SHA for 0f39d48 - Browse repository at this point
Copy the full SHA 0f39d48View commit details -
Configuration menu - View commit details
-
Copy full SHA for e98589b - Browse repository at this point
Copy the full SHA e98589bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 107ece0 - Browse repository at this point
Copy the full SHA 107ece0View commit details -
Configuration menu - View commit details
-
Copy full SHA for c4bebea - Browse repository at this point
Copy the full SHA c4bebeaView commit details -
Configuration menu - View commit details
-
Copy full SHA for f4f1a62 - Browse repository at this point
Copy the full SHA f4f1a62View commit details -
Configuration menu - View commit details
-
Copy full SHA for 452c7cd - Browse repository at this point
Copy the full SHA 452c7cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3f219c7 - Browse repository at this point
Copy the full SHA 3f219c7View commit details -
Merge pull request #15 from vilterp/pv-alloc-profile-allocs-missed
alloc profile: print percentage of allocs missed
Configuration menu - View commit details
-
Copy full SHA for b25005a - Browse repository at this point
Copy the full SHA b25005aView commit details
Commits on Jan 12, 2022
-
Configuration menu - View commit details
-
Copy full SHA for bf08801 - Browse repository at this point
Copy the full SHA bf08801View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8ec194b - Browse repository at this point
Copy the full SHA 8ec194bView commit details -
Merge pull request #18 from vilterp/pv-alloc-profile-record-strings
alloc profile: record string allocs
Configuration menu - View commit details
-
Copy full SHA for 41f169c - Browse repository at this point
Copy the full SHA 41f169cView commit details -
alloc profiler: avoid divide-by-zero errors (#19)
* avoid divide-by-zero errors when printing error message
Configuration menu - View commit details
-
Copy full SHA for 3fc80ac - Browse repository at this point
Copy the full SHA 3fc80acView commit details
Commits on Jan 13, 2022
-
add paragraph about missed allocations
Co-authored-by: Nathan Daly <NHDaly@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9465507 - Browse repository at this point
Copy the full SHA 9465507View commit details
Commits on Jan 18, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 76ad6f0 - Browse repository at this point
Copy the full SHA 76ad6f0View commit details -
Configuration menu - View commit details
-
Copy full SHA for dd693cc - Browse repository at this point
Copy the full SHA dd693ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4deaf6c - Browse repository at this point
Copy the full SHA 4deaf6cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 13a85de - Browse repository at this point
Copy the full SHA 13a85deView commit details -
Configuration menu - View commit details
-
Copy full SHA for 68e1a15 - Browse repository at this point
Copy the full SHA 68e1a15View commit details -
Configuration menu - View commit details
-
Copy full SHA for b5f636a - Browse repository at this point
Copy the full SHA b5f636aView commit details
Commits on Jan 19, 2022
-
Configuration menu - View commit details
-
Copy full SHA for b16b610 - Browse repository at this point
Copy the full SHA b16b610View commit details