Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alloc profiler: get stacks for pool allocs via wrapper function #17

Closed

Conversation

vilterp
Copy link
Owner

@vilterp vilterp commented Jan 12, 2022

merges into JuliaLang#42768

Add a wrapper around jl_gc_pool_alloc to allow us to instrument it.

managed_malloc taken care of in #20

Before

image

*: instrumented; red: currently missed

graphviz
digraph G {
  node [shape=box];

  generated_code [label="<generated code>"];

  new_array;
  jl_gc_alloc_string [label="jl_gc_alloc_string *"];
  jl_gc_alloc [label="jl_gc_alloc *"];
  jl_managed_malloc [label="jl_managed_malloc *"]; // https://github.com/vilterp/julia/pull/20
  jl_gc_pool_alloc;
  jl_gc_big_alloc;

  generated_code -> jl_gc_alloc_string;
  generated_code -> new_array;
  generated_code -> jl_gc_alloc;
  generated_code -> jl_gc_big_alloc [color=red];
  generated_code -> jl_gc_pool_alloc [color=red];

  jl_gc_alloc -> jl_gc_pool_alloc;
  jl_gc_alloc -> jl_gc_big_alloc;
  new_array -> jl_gc_alloc;
  new_array -> jl_managed_malloc;
  jl_gc_alloc_string -> jl_gc_pool_alloc;
  jl_gc_alloc_string -> jl_gc_big_alloc;
}

After

image

*: instrumented; red: currently missed

graphviz ```graphviz digraph G { node [shape=box];

generated_code [label=""];

new_array;
jl_gc_alloc_string [label="jl_gc_alloc_string *"];
jl_gc_alloc [label="jl_gc_alloc *"];
jl_managed_malloc [label="jl_managed_malloc *"]; // #20
jl_gc_pool_alloc;
jl_gc_pool_alloc_outer [fillcolor=lightgreen style=filled label="jl_gc_pool_alloc_outer *"];
jl_gc_big_alloc;

generated_code -> jl_gc_alloc_string;
generated_code -> new_array;
generated_code -> jl_gc_alloc;
generated_code -> jl_gc_big_alloc [color=red];
generated_code -> jl_gc_pool_alloc_outer;
jl_gc_pool_alloc_outer -> jl_gc_pool_alloc;

jl_gc_alloc -> jl_gc_pool_alloc;
jl_gc_alloc -> jl_gc_big_alloc;
new_array -> jl_gc_alloc;
new_array -> jl_managed_malloc;
jl_gc_alloc_string -> jl_gc_pool_alloc;
jl_gc_alloc_string -> jl_gc_big_alloc;
}

</details>

vchuravy and others added 30 commits January 6, 2022 15:47
Prevents loading of `libjulia-codegen` into `opt`.

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
Regression introduced by ca11c2f (when tests fail or are interrupted)
See comment in diff for explanation. This fixes test/stacktraces.jl
on aarch64 macOS 12, and according to an OpenJDK issue where they
ran into the same problem, https://git.openjdk.java.net/jdk/pull/6193,
probably also x86_64 macOS 12.
Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
In a similar spirit to JuliaLang#43587, this commit introduces error check for
`setfield!`. We can bail out from inference if we can prove either of:
- the object is not mutable type
- the object is `Module` object
- the value being assigned is incompatible with the declared type of
  object field

This commit also adds the throwness check for `setfield!` (i.e. `setfield!_nothrow`).
This throwness check won't be used in the current native compilation
pipeline since `setfield!` call can't be eliminated even if we can prove
that it never throws. But this throwness check would be used by
EscapeAnalysis.jl integration and so I'd like to include it in Base.
…JuliaLang#43587)

Obvious errors are usually caught in higher places like `convert`, but
it's better to have error checks at each builtin level in order to enable
early bail out from errorneous code compilation (when somehow it does not
rely on common abstraction). These checks are also useful for third
consumers like JET.
…uliaLang#43684)

Previously, we had the machinery to load per-UUID, but some tooling
needs to be able to load all preferences (without loading
`Preferences.jl`), so it's better to support being able to load
everything from just within Base.
…FO` (for e.g. when Git is not available) (JuliaLang#43717)

* `base/version_git.sh`: fix the default value for `Base.GIT_VERSION_INFO` (for e.g. when Git is not available)

* Use single-quotes to avoid the need to backslash-escape the double quotes
4c45f29 had reintroduced
a use of getNumArgOperands(), which was already slated for
removal in (at least) LLVM 12.
This shouldn't matter on any platform with a sensible ABI; just
happened across this when tracking down some other issue.
Also makes it clear to the reader that signals-mach.c uses
a different approach entirely (which almost tripped me up
the first time I was reading through this part of the codebase).
@vilterp vilterp marked this pull request as ready for review January 18, 2022 04:53
bkamins and others added 3 commits January 18, 2022 13:33
…ng#43426)

We could also consider adding `defalg(v::AbstractArray{<:Number}) = DEFAULT_UNSTABLE`,
but it is unlikely that someone will want to do `Union` of `<:Number`
and something other than `Missing` that still would support comparison.

Relevant for JuliaStrings/InlineStrings.jl#21
(and other custom types that will want to add `defalg` support allowing
for `Union` with `Missing`).
…3830)

This was open-coded inside inlining, but the effect-free modeling
code should be with the other builtins (in preparation of calling
it from more places).
@NHDaly NHDaly mentioned this pull request Jan 18, 2022
7 tasks
@NHDaly NHDaly force-pushed the pv-alloc-profile-wrap-pool-alloc branch from 85d4797 to 5c6ae25 Compare January 18, 2022 19:44
@vilterp vilterp changed the title alloc profiler: attempt #2 at getting stacks for all allocs alloc profiler: get stacks for pool allocs via wrapper function Jan 18, 2022
Comment on lines +1200 to +1201
JL_DLLEXPORT jl_value_t *jl_gc_pool_alloc_outer(jl_ptls_t ptls, int pool_offset,
int osize)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main comment is about trying to find a more informative name for this, like maybe:

  • jl_gc_pool_alloc_from_gen
  • jl_gc_pool_alloc_from_llvm_gen
  • jl_gc_pool_alloc_from_generated...
  • jl_gc_pool_alloc_that_will_be_called_from_generated_code... i dunno...

Blech naming is hard

Copy link
Collaborator

@NHDaly NHDaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. So, I think this looks great, totally solves the problem, and makes sense! 👍

I do think it'd be good to get a second opinion. Thanks for all the reviews of late, @vchuravy -- would you be able to have a look here too? The PR description explains very well what we did here. ❤️

@NHDaly NHDaly requested a review from vchuravy January 18, 2022 21:41
@@ -1196,6 +1196,16 @@ static NOINLINE jl_taggedvalue_t *add_page(jl_gc_pool_t *p) JL_NOTSAFEPOINT
return fl;
}

// called into by LLVM-generated code
JL_DLLEXPORT jl_value_t *jl_gc_pool_alloc_outer(jl_ptls_t ptls, int pool_offset,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer keeping the same name here, and renaming the inner internal implementation, which then also shouldn't be exported (and ideally inlined into this function).

Maybe call that one then jl_gc_pool_alloc_impl

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We talked about inlining it too, but we weren't sure about how to inline it only into this function..

If we mark that one inline, it'll end up inlining into the normal callers of it, though.. So the C functions (indicated in Pete's graph, above) that calls jl_gc_pool_alloc would now call jl_gc_pool_alloc_impl instead, and then it would be inlined.

So it'd become inlined into jl_gc_alloc_ and jl_gc_alloc_string. Maybe that's okay?

Or would you want us to create yet a third separate wrapper function that gets called by those? So we would have three functions:

  • jl_gc_pool_alloc_from_generated: called from generated code
  • jl_gc_pool_alloc: called from jl_gc_alloc_ and jl_gc_alloc_string
  • jl_gc_pool_alloc_impl: called from jl_gc_pool_alloc
    ? Or is that too weird? And the extra inlining is actually fine?

N5N3 and others added 7 commits January 18, 2022 14:43
* Define `extrema` using `mapreduce`; support `init`

* Fix tests for SparseArrays

* API clean and export `extrema!`

* Re-implement `reducedim_init` for extrema

* Merge `master` to pull in JuliaSparse/SparseArrays.jl#63

* Mark `BigInt` tests as broken

Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>
Co-authored-by: Simeon Schaub <simeondavidschaub99@gmail.com>
Co-authored-by: Takafumi Arakaki <aka.tkf@gmail.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>
…#43812)

When `tmerge` is applied to `Const`/`PartialStruct` and a field is
`Const` in one of the types and `Union{}` (i.e. undef) in the other,
the resulting field type must not be `Const` (as the resursively called
`tmerge` produces).

Fixes JuliaLang#43784.
## Overview

Record the type and stack of every allocation (or only at a given sample interval), and return as Julia objects.

Alternate approach to existing alloc profiler PR: JuliaLang#33467
Complementary to garbage profiler PR: JuliaLang#42658 (maybe there's some nice way to meld them)

This may be reinventing the wheel from JuliaLang#33467, but I'm not sure why that one needs stuff like LLVM passes. I mimicked some stuff from it, but this was my attempt to get something up and running. Could easily be missing stuff.

## Usage:

```julia
using Profile.Allocs
res = Allocs.@Profile sample_rate=0.001 my_func()
prof = Allocs.fetch()
# do something with `prof`
```

See also: JuliaPerf/PProf.jl#46 for support for visualizing these.

Co-authored-by: Nathan Daly <nhdaly@gmail.com>
@vilterp vilterp force-pushed the pv-alloc-profile-wrap-pool-alloc branch from 6bed933 to d0f5ce0 Compare January 19, 2022 20:57
@vilterp
Copy link
Owner Author

vilterp commented Jan 19, 2022

Closed in favor of JuliaLang#43868 cuz I couldn't change the base to point at JuliaLang/julia

@vilterp vilterp closed this Jan 19, 2022
vilterp pushed a commit that referenced this pull request Nov 17, 2022
When calling `jl_error()` or `jl_errorf()`, we must check to see if we
are so early in the bringup process that it is dangerous to attempt to
construct a backtrace because the data structures used to provide line
information are not properly setup.

This can be easily triggered by running:

```
julia -C invalid
```

On an `i686-linux-gnu` build, this will hit the "Invalid CPU Name"
branch in `jitlayers.cpp`, which calls `jl_errorf()`.  This in turn
calls `jl_throw()`, which will eventually call `jl_DI_for_fptr` as part
of the backtrace printing process, which fails as the object maps are
not fully initialized.  See the below `gdb` stacktrace for details:

```
$ gdb -batch -ex 'r' -ex 'bt' --args ./julia -C invalid
...
fatal: error thrown and no exception handler available.
ErrorException("Invalid CPU name "invalid".")

Thread 1 "julia" received signal SIGSEGV, Segmentation fault.
0xf75bd665 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo>, std::_Select1st<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> >, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__k=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h:1277
1277    /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h: No such file or directory.
 #0  0xf75bd665 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo>, std::_Select1st<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> >, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__k=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h:1277
 #1  std::map<unsigned int, JITDebugInfoRegistry::ObjectInfo, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__x=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_map.h:1258
 #2  jl_DI_for_fptr (fptr=4155049385, symsize=symsize@entry=0xffffcfa8, slide=slide@entry=0xffffcfa0, Section=Section@entry=0xffffcfb8, context=context@entry=0xffffcf94) at /cache/build/default-amdci5-4/julialang/julia-master/src/debuginfo.cpp:1181
 #3  0xf75c056a in jl_getFunctionInfo_impl (frames_out=0xffffd03c, pointer=4155049385, skipC=0, noInline=0) at /cache/build/default-amdci5-4/julialang/julia-master/src/debuginfo.cpp:1210
 #4  0xf7a6ca98 in jl_print_native_codeloc (ip=4155049385) at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:636
 #5  0xf7a6cd54 in jl_print_bt_entry_codeloc (bt_entry=0xf0798018) at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:657
 #6  jlbacktrace () at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:1090
 #7  0xf7a3cd2b in ijl_no_exc_handler (e=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:605
 #8  0xf7a3d10a in throw_internal (ct=ct@entry=0xf070c010, exception=<optimized out>, exception@entry=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:638
 #9  0xf7a3d330 in ijl_throw (e=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:654
 #10 0xf7a905aa in ijl_errorf (fmt=fmt@entry=0xf7647cd4 "Invalid CPU name \"%s\".") at /cache/build/default-amdci5-4/julialang/julia-master/src/rtutils.c:77
 #11 0xf75a4b22 in (anonymous namespace)::createTargetMachine () at /cache/build/default-amdci5-4/julialang/julia-master/src/jitlayers.cpp:823
 #12 JuliaOJIT::JuliaOJIT (this=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/jitlayers.cpp:1044
 #13 0xf7531793 in jl_init_llvm () at /cache/build/default-amdci5-4/julialang/julia-master/src/codegen.cpp:8585
 #14 0xf75318a8 in jl_init_codegen_impl () at /cache/build/default-amdci5-4/julialang/julia-master/src/codegen.cpp:8648
 #15 0xf7a51a52 in jl_restore_system_image_from_stream (f=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2131
 #16 0xf7a55c03 in ijl_restore_system_image_data (buf=0xe859c1c0 <jl_system_image_data> "8'\031\003", len=125161105) at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2184
 #17 0xf7a55cf9 in jl_load_sysimg_so () at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:424
 #18 ijl_restore_system_image (fname=0x80a0900 "/build/bk_download/julia-d78fdad601/lib/julia/sys.so") at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2157
 #19 0xf7a3bdfc in _finish_julia_init (rel=rel@entry=JL_IMAGE_JULIA_HOME, ct=<optimized out>, ptls=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/init.c:741
 #20 0xf7a3c8ac in julia_init (rel=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/init.c:728
 #21 0xf7a7f61d in jl_repl_entrypoint (argc=<optimized out>, argv=0xffffddf4) at /cache/build/default-amdci5-4/julialang/julia-master/src/jlapi.c:705
 #22 0x080490a7 in main (argc=3, argv=0xffffddf4) at /cache/build/default-amdci5-4/julialang/julia-master/cli/loader_exe.c:59
```

To prevent this, we simply avoid calling `jl_errorf` this early in the
process, punting the problem to a later PR that can update guard
conditions within `jl_error*`.
vilterp pushed a commit that referenced this pull request Nov 18, 2022
When calling `jl_error()` or `jl_errorf()`, we must check to see if we
are so early in the bringup process that it is dangerous to attempt to
construct a backtrace because the data structures used to provide line
information are not properly setup.

This can be easily triggered by running:

```
julia -C invalid
```

On an `i686-linux-gnu` build, this will hit the "Invalid CPU Name"
branch in `jitlayers.cpp`, which calls `jl_errorf()`.  This in turn
calls `jl_throw()`, which will eventually call `jl_DI_for_fptr` as part
of the backtrace printing process, which fails as the object maps are
not fully initialized.  See the below `gdb` stacktrace for details:

```
$ gdb -batch -ex 'r' -ex 'bt' --args ./julia -C invalid
...
fatal: error thrown and no exception handler available.
ErrorException("Invalid CPU name "invalid".")

Thread 1 "julia" received signal SIGSEGV, Segmentation fault.
0xf75bd665 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo>, std::_Select1st<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> >, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__k=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h:1277
1277    /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h: No such file or directory.
 #0  0xf75bd665 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo>, std::_Select1st<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> >, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__k=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h:1277
 #1  std::map<unsigned int, JITDebugInfoRegistry::ObjectInfo, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__x=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_map.h:1258
 #2  jl_DI_for_fptr (fptr=4155049385, symsize=symsize@entry=0xffffcfa8, slide=slide@entry=0xffffcfa0, Section=Section@entry=0xffffcfb8, context=context@entry=0xffffcf94) at /cache/build/default-amdci5-4/julialang/julia-master/src/debuginfo.cpp:1181
 #3  0xf75c056a in jl_getFunctionInfo_impl (frames_out=0xffffd03c, pointer=4155049385, skipC=0, noInline=0) at /cache/build/default-amdci5-4/julialang/julia-master/src/debuginfo.cpp:1210
 #4  0xf7a6ca98 in jl_print_native_codeloc (ip=4155049385) at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:636
 #5  0xf7a6cd54 in jl_print_bt_entry_codeloc (bt_entry=0xf0798018) at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:657
 #6  jlbacktrace () at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:1090
 #7  0xf7a3cd2b in ijl_no_exc_handler (e=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:605
 #8  0xf7a3d10a in throw_internal (ct=ct@entry=0xf070c010, exception=<optimized out>, exception@entry=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:638
 #9  0xf7a3d330 in ijl_throw (e=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:654
 #10 0xf7a905aa in ijl_errorf (fmt=fmt@entry=0xf7647cd4 "Invalid CPU name \"%s\".") at /cache/build/default-amdci5-4/julialang/julia-master/src/rtutils.c:77
 #11 0xf75a4b22 in (anonymous namespace)::createTargetMachine () at /cache/build/default-amdci5-4/julialang/julia-master/src/jitlayers.cpp:823
 #12 JuliaOJIT::JuliaOJIT (this=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/jitlayers.cpp:1044
 #13 0xf7531793 in jl_init_llvm () at /cache/build/default-amdci5-4/julialang/julia-master/src/codegen.cpp:8585
 #14 0xf75318a8 in jl_init_codegen_impl () at /cache/build/default-amdci5-4/julialang/julia-master/src/codegen.cpp:8648
 #15 0xf7a51a52 in jl_restore_system_image_from_stream (f=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2131
 #16 0xf7a55c03 in ijl_restore_system_image_data (buf=0xe859c1c0 <jl_system_image_data> "8'\031\003", len=125161105) at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2184
 #17 0xf7a55cf9 in jl_load_sysimg_so () at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:424
 #18 ijl_restore_system_image (fname=0x80a0900 "/build/bk_download/julia-d78fdad601/lib/julia/sys.so") at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2157
 #19 0xf7a3bdfc in _finish_julia_init (rel=rel@entry=JL_IMAGE_JULIA_HOME, ct=<optimized out>, ptls=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/init.c:741
 #20 0xf7a3c8ac in julia_init (rel=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/init.c:728
 #21 0xf7a7f61d in jl_repl_entrypoint (argc=<optimized out>, argv=0xffffddf4) at /cache/build/default-amdci5-4/julialang/julia-master/src/jlapi.c:705
 #22 0x080490a7 in main (argc=3, argv=0xffffddf4) at /cache/build/default-amdci5-4/julialang/julia-master/cli/loader_exe.c:59
```

To prevent this, we simply avoid calling `jl_errorf` this early in the
process, punting the problem to a later PR that can update guard
conditions within `jl_error*`.

(cherry picked from commit 21ab24e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.