Description
openedon Apr 18, 2024
So we currently store absolut paths in debuginfo for Julia and it's library. Locally this might be fine, but for our build artifacts this is rather pointless.
@KristofferC raised this recently as:
jl_apply at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-11/src/./julia.h:2154 [inlined]
jl_f__call_latest at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-11/src/builtins.c:875
The buildpath of /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-11 is long and at least for me unecessary.
There are two parts of this.
- Runtime and libraries
- libjulia & co
- libLLVM & co (this we would need to fix in BinaryBuilder)
- Code coming from Julia
An example for 2 comes from:
julia> Profile.print()
Overhead ╎ [+additional indent] Count File:Line; Function
=========================================================
╎17 @Base/client.jl:552; _start()
╎ 17 @Base/client.jl:333; exec_options(opts::Base.JLOptions)
╎ 17 @Base/client.jl:416; run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
╎ 17 @Base/essentials.jl:889; invokelatest
╎ 17 @Base/essentials.jl:892; #invokelatest#2
╎ 17 @Base/client.jl:432; (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
╎ ╎ 17 …/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375; run_repl(repl::REPL.AbstractREPL, consumer::Any)
We already do this for Base so maybe we can change that latter path to @REPL?
The bigger challenge here is to make sure that system profilers, "click on path" in REPL and other things keep working.
So we might need to remap files to local directories for presentation.
While discussing with @KristofferC I noticed that GCC/Clang support remapping paths using -ffile-prefix-map.
- https://forums.swift.org/t/standardizing-support-for-path-remappings-hermetic-builds-and-indexing/56713
- https://maskray.me/blog/2022-08-14-dwarf-in-reproducible-builds
I locally tried the following patch:
diff --git a/Make.inc b/Make.inc
index 56558e17bb..6e56ceca65 100644
--- a/Make.inc
+++ b/Make.inc
@@ -509,7 +509,7 @@ DEBUGFLAGS_COMMON := -O0 -DJL_DEBUG_BUILD -fstack-protector
DEBUGFLAGS_CLANG := $(DEBUGFLAGS_COMMON) -g
DEBUGFLAGS_GCC := $(DEBUGFLAGS_COMMON) -ggdb2
-SHIPFLAGS_COMMON := -O3
+SHIPFLAGS_COMMON := -O3 -ffile-prefix-map=$(BUILDDIR)/= -ffile-prefix-map=$(JULIAHOME)/=
SHIPFLAGS_CLANG := $(SHIPFLAGS_COMMON) -g
SHIPFLAGS_GCC := $(SHIPFLAGS_COMMON) -ggdb2 -falign-functions
and the test-case:
julia> using Profile
julia> @profile 1+1
2
julia> @profile rand(5);
julia> Profile.print(C=true)
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 129 src/gf.c:2571; jl_compile_method_internal
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 129 src/jitlayers.cpp:483; jl_compile_codeinst_impl
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 src/jitlayers.cpp:219; _jl_compile_codeinst
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 src/codegen.cpp:9648; jl_emit_codeinst
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 src/codegen.cpp:9560; jl_emit_code
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 src/codegen.cpp:9225; emit_function
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 src/codegen.cpp:5736; emit_ssaval_assign
╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 src/codegen.cpp:6186; emit_expr
Opening this as an issue instead of a PR since I suspect this will need a larger conversation that also includes changes to BinaryBuilder.