bolt segfault on python aarch64 build

[belm0]

This segfault is observed when building python 3.12 (seen on 3.12.4 and 3.12.5).

1336.7 BOLT-INFO: Target architecture: aarch64
1336.7 BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
1336.7 BOLT-INFO: first alloc address is 0x400000
1336.7 BOLT-INFO: enabling relocation mode
1336.7 BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
1337.0 BOLT-INFO: pre-processing profile using branch profile reader
1338.2 BOLT-INFO: number of removed linker-inserted veneers: 0
1338.2 BOLT-INFO: 3957 out of 5934 functions in the binary (66.7%) have non-empty execution profile
1338.2 BOLT-INFO: 91 functions with profile could not be optimized
1338.2 BOLT-INFO: profile for 1 objects was ignored
1338.2 BOLT-INFO: removed 200 empty blocks
1338.3 BOLT-INFO: ICF folded 163 out of 6294 functions in 4 passes. 0 functions had jump tables.
1338.3 BOLT-INFO: Removing all identical functions will save 4.83 KB of code space. Folded functions were called 26607714 times based on profile.
1338.5 BOLT-INFO: ICP Total indirect calls = 481441560, 263 callsites cover 99% of all indirect calls
1338.6  #0 0x0000aaaac22c8c20 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1608c20)
1338.6  #1 0x0000aaaac22c6dc0 llvm::sys::RunSignalHandlers() (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1606dc0)
1338.6  #2 0x0000aaaac22c9478 SignalHandler(int) Signals.cpp:0:0
1338.6  #3 0x0000ffff84c417a0 (linux-vdso.so.1+0x7a0)
1338.6  #4 0x0000aaaac27171ac llvm::bolt::IndirectCallPromotion::rewriteCall(llvm::bolt::BinaryBasicBlock&, llvm::MCInst const&, std::vector<std::pair<llvm::MCSymbol*, std::vector<llvm::MCInst, std::allocator<llvm::MCInst>>>, std::allocator<std::pair<llvm::MCSymbol*, std::vector<llvm::MCInst, std::allocator<llvm::MCInst>>>>>&&, std::vector<llvm::MCInst*, std::allocator<llvm::MCInst*>> const&) const (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a571ac)
1338.6  #5 0x0000aaaac271b124 llvm::bolt::IndirectCallPromotion::runOnFunctions(llvm::bolt::BinaryContext&) (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a5b124)
1338.6  #6 0x0000aaaac2355d44 llvm::bolt::BinaryFunctionPassManager::runPasses() (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1695d44)
1338.6  #7 0x0000aaaac2359914 llvm::bolt::BinaryFunctionPassManager::runAllPasses(llvm::bolt::BinaryContext&) (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1699914)
1338.6  #8 0x0000aaaac2309410 llvm::bolt::RewriteInstance::run() (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1649410)
1338.6  #9 0x0000aaaac0fe18d0 main (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x3218d0)
1338.6 #10 0x0000ffff84745e10 __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x20e10)
1338.6 #11 0x0000aaaac0fdfd68 _start (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x31fd68)
1338.6 PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
1338.6 Stack dump:
1338.6 0.	Program arguments: /root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -inline-all -split-eh -reorder-functions-use-hot-size -peepholes=none -jump-tables=aggressive -inline-ap -indirect-call-promotion=all -dyno-stats -use-gnu-stack -frame-opt=hot
1338.6 Segmentation fault

[llvmbot]

@llvm/issue-subscribers-bolt

Author: John Belmonte (belm0)

This segfault is observed when building python 3.12 (seen on 3.12.4 and 3.12.5).

1336.7 BOLT-INFO: Target architecture: aarch64
1336.7 BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
1336.7 BOLT-INFO: first alloc address is 0x400000
1336.7 BOLT-INFO: enabling relocation mode
1336.7 BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
1337.0 BOLT-INFO: pre-processing profile using branch profile reader
1338.2 BOLT-INFO: number of removed linker-inserted veneers: 0
1338.2 BOLT-INFO: 3957 out of 5934 functions in the binary (66.7%) have non-empty execution profile
1338.2 BOLT-INFO: 91 functions with profile could not be optimized
1338.2 BOLT-INFO: profile for 1 objects was ignored
1338.2 BOLT-INFO: removed 200 empty blocks
1338.3 BOLT-INFO: ICF folded 163 out of 6294 functions in 4 passes. 0 functions had jump tables.
1338.3 BOLT-INFO: Removing all identical functions will save 4.83 KB of code space. Folded functions were called 26607714 times based on profile.
1338.5 BOLT-INFO: ICP Total indirect calls = 481441560, 263 callsites cover 99% of all indirect calls
1338.6  #<!-- -->0 0x0000aaaac22c8c20 llvm::sys::PrintStackTrace(llvm::raw_ostream&amp;, int) (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1608c20)
1338.6  #<!-- -->1 0x0000aaaac22c6dc0 llvm::sys::RunSignalHandlers() (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1606dc0)
1338.6  #<!-- -->2 0x0000aaaac22c9478 SignalHandler(int) Signals.cpp:0:0
1338.6  #<!-- -->3 0x0000ffff84c417a0 (linux-vdso.so.1+0x7a0)
1338.6  #<!-- -->4 0x0000aaaac27171ac llvm::bolt::IndirectCallPromotion::rewriteCall(llvm::bolt::BinaryBasicBlock&amp;, llvm::MCInst const&amp;, std::vector&lt;std::pair&lt;llvm::MCSymbol*, std::vector&lt;llvm::MCInst, std::allocator&lt;llvm::MCInst&gt;&gt;&gt;, std::allocator&lt;std::pair&lt;llvm::MCSymbol*, std::vector&lt;llvm::MCInst, std::allocator&lt;llvm::MCInst&gt;&gt;&gt;&gt;&gt;&amp;&amp;, std::vector&lt;llvm::MCInst*, std::allocator&lt;llvm::MCInst*&gt;&gt; const&amp;) const (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a571ac)
1338.6  #<!-- -->5 0x0000aaaac271b124 llvm::bolt::IndirectCallPromotion::runOnFunctions(llvm::bolt::BinaryContext&amp;) (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a5b124)
1338.6  #<!-- -->6 0x0000aaaac2355d44 llvm::bolt::BinaryFunctionPassManager::runPasses() (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1695d44)
1338.6  #<!-- -->7 0x0000aaaac2359914 llvm::bolt::BinaryFunctionPassManager::runAllPasses(llvm::bolt::BinaryContext&amp;) (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1699914)
1338.6  #<!-- -->8 0x0000aaaac2309410 llvm::bolt::RewriteInstance::run() (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1649410)
1338.6  #<!-- -->9 0x0000aaaac0fe18d0 main (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x3218d0)
1338.6 #<!-- -->10 0x0000ffff84745e10 __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x20e10)
1338.6 #<!-- -->11 0x0000aaaac0fdfd68 _start (/root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x31fd68)
1338.6 PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
1338.6 Stack dump:
1338.6 0.	Program arguments: /root/llvm-project/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -inline-all -split-eh -reorder-functions-use-hot-size -peepholes=none -jump-tables=aggressive -inline-ap -indirect-call-promotion=all -dyno-stats -use-gnu-stack -frame-opt=hot
1338.6 Segmentation fault

[paschalis-mpeis]

Hey @belm0 ,

Some of the passes used contain unimplemented functionality, which is most likely why you see the crash.
In this case, Indirect Call Promotion, and probably Inliner could have crashed later on.

For a start, can you be even more conservative, remove all the below, and optionally report whether you were able to generate a BOLTed binary?"

-jump-tables=aggressive -indirect-call-promotion=all
-inline-ap -inline-all
-use-gnu-stack -frame-opt=hot

[iovxw]

Hi, got same error on python 3.13:

log

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 4 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
BOLT-INFO: ICP Total indirect calls = 2518250190, 314 callsites cover 99% of all indirect calls
 #0 0x0000aaaad83b8c20 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1608c20)
 #1 0x0000aaaad83b6dc0 llvm::sys::RunSignalHandlers() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1606dc0)
 #2 0x0000aaaad83b9478 SignalHandler(int) Signals.cpp:0:0
 #3 0x0000ffffb05157bc (linux-vdso.so.1+0x7bc)
 #4 0x0000aaaad88071ac llvm::bolt::IndirectCallPromotion::rewriteCall(llvm::bolt::BinaryBasicBlock&, llvm::MCInst const&, std::vector<std::pair<llvm::MCSymbol*, std::vector<llvm::MCInst, std::allocator<llvm::MCInst>>>, std::allocator<std::pair<llvm::MCSymbol*, std::vector<llvm::MCInst, std::allocator<llvm::MCInst>>>>>&&, std::vector<llvm::MCInst*, std::allocator<llvm::MCInst*>> const&) const (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a571ac)
 #5 0x0000aaaad880b124 llvm::bolt::IndirectCallPromotion::runOnFunctions(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a5b124)
 #6 0x0000aaaad8445d44 llvm::bolt::BinaryFunctionPassManager::runPasses() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1695d44)
 #7 0x0000aaaad8449914 llvm::bolt::BinaryFunctionPassManager::runAllPasses(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1699914)
 #8 0x0000aaaad83f9410 llvm::bolt::RewriteInstance::run() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1649410)
 #9 0x0000aaaad70d18d0 main (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x3218d0)
#10 0x0000ffffaff37740 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#11 0x0000ffffaff37818 call_init ./csu/../csu/libc-start.c:128:20
#12 0x0000ffffaff37818 __libc_start_main ./csu/../csu/libc-start.c:347:5
#13 0x0000aaaad70cfd68 _start (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x31fd68)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -inline-all -split-eh -reorder-functions-use-hot-size -peepholes=none -jump-tables=aggressive -inline-ap -indirect-call-promotion=all -dyno-stats -use-gnu-stack -frame-opt=hot
Segmentation fault

In this case, Indirect Call Promotion, and probably Inliner could have crashed later on.

For a start, can you be even more conservative, remove all the below, and optionally report whether you were able to generate a BOLTed binary?"

I can confirm it can be fixed by remove -indirect-call-promotion=all -inline-all and -frame-opt=hot

$ /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -jump-tables=aggressive -inline-ap -use-gnu-stack

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 5 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
BOLT-INFO: basic block reordering modified layout of 2372 functions (55.79% of profiled, 30.04% of total)
BOLT-INFO: 578 Functions were reordered by LoopInversionPass
BOLT-INFO: hfsort+ reduced the number of chains from 4197 to 2213
BOLT-INFO: program-wide dynostats after all optimizations before SCTC and FOP:

         24573349557 : executed forward branches
          3063021853 : taken forward branches
          5302883293 : executed backward branches
          3694579499 : taken backward branches
          1067304819 : executed unconditional branches
          3574484778 : all function calls
          2897719906 : indirect calls
           125410746 : PLT calls
        162459659259 : executed instructions
         27984269255 : executed load instructions
                   0 : executed store instructions
                   0 : taken jump table branches
                   0 : taken unknown indirect branches
         30943537669 : total branches
          7824906171 : taken branches
         23118631498 : non-taken conditional branches
          6757601352 : taken conditional branches
         29876232850 : all conditional branches
                   0 : linker-inserted veneer calls

         24109456738 : executed forward branches (-1.9%)
          1930256656 : taken forward branches (-37.0%)
          5766776112 : executed backward branches (+8.7%)
          3485693463 : taken backward branches (-5.7%)
           778341212 : executed unconditional branches (-27.1%)
          3574484778 : all function calls (=)
          2897719906 : indirect calls (=)
           125410746 : PLT calls (=)
        160601637149 : executed instructions (-1.1%)
         27984269255 : executed load instructions (=)
                   0 : executed store instructions (=)
                   0 : taken jump table branches (=)
                   0 : taken unknown indirect branches (=)
         30654574062 : total branches (-0.9%)
          6194291331 : taken branches (-20.8%)
         24460282731 : non-taken conditional branches (+5.8%)
          5415950119 : taken conditional branches (-19.9%)
         29876232850 : all conditional branches (=)
                   0 : linker-inserted veneer calls (=)

BOLT-INFO: Starting stub-insertion pass
BOLT-INFO: Inserted 22208 stubs in the hot area and 10316 stubs in the cold area. Shared 0 times, iterated 3 times.
BOLT-INFO: padding code to 0xe00000 to accommodate hot text
BOLT-INFO: setting __hot_start to 0xa00000
BOLT-INFO: setting __hot_end to 0xc0cce8

And the error log of with -inline-all:

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: creating new program header table at address 0xa00000, offset 0x800000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 4 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
 #0 0x0000aaaae0ac8c20 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1608c20)
 #1 0x0000aaaae0ac6dc0 llvm::sys::RunSignalHandlers() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1606dc0)
 #2 0x0000aaaae0ac9478 SignalHandler(int) Signals.cpp:0:0
 #3 0x0000ffff959d67bc (linux-vdso.so.1+0x7bc)
 #4 0x0000aaaae1004a94 (anonymous namespace)::AArch64MCPlusBuilder::replaceImmWithSymbolRef(llvm::MCInst&, llvm::MCSymbol const*, long, llvm::MCContext*, long&, unsigned long) const AArch64MCPlusBuilder.cpp:0:0
 #5 0x0000aaaae0f22b88 llvm::bolt::getInliningInfo(llvm::bolt::BinaryFunction const&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a62b88)
 #6 0x0000aaaae0f22ed8 llvm::bolt::Inliner::findInliningCandidates(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a62ed8)
 #7 0x0000aaaae0f24a44 llvm::bolt::Inliner::runOnFunctions(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a64a44)
 #8 0x0000aaaae0b55d44 llvm::bolt::BinaryFunctionPassManager::runPasses() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1695d44)
 #9 0x0000aaaae0b59914 llvm::bolt::BinaryFunctionPassManager::runAllPasses(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1699914)
#10 0x0000aaaae0b09410 llvm::bolt::RewriteInstance::run() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1649410)
#11 0x0000aaaadf7e18d0 main (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x3218d0)
#12 0x0000ffff953f7740 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#13 0x0000ffff953f7818 call_init ./csu/../csu/libc-start.c:128:20
#14 0x0000ffff953f7818 __libc_start_main ./csu/../csu/libc-start.c:347:5
#15 0x0000aaaadf7dfd68 _start (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x31fd68)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -inline-all
Segmentation fault

-frame-opt=hot got segfault with no stack trace:

$ /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -frame-opt=hot

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: creating new program header table at address 0xa00000, offset 0x800000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 6 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
BOLT-INFO: basic block reordering modified layout of 2372 functions (55.79% of profiled, 30.04% of total)
BOLT-INFO: 578 Functions were reordered by LoopInversionPass
BOLT-INFO: hfsort+ reduced the number of chains from 4197 to 2213
BOLT-INFO: program-wide dynostats after all optimizations before SCTC and FOP:

         24573349557 : executed forward branches
          3063021853 : taken forward branches
          5302883293 : executed backward branches
          3694579499 : taken backward branches
          1067304819 : executed unconditional branches
          3574484778 : all function calls
          2897719906 : indirect calls
           125410746 : PLT calls
        162459659259 : executed instructions
         27984269255 : executed load instructions
                   0 : executed store instructions
                   0 : taken jump table branches
                   0 : taken unknown indirect branches
         30943537669 : total branches
          7824906171 : taken branches
         23118631498 : non-taken conditional branches
          6757601352 : taken conditional branches
         29876232850 : all conditional branches
                   0 : linker-inserted veneer calls

         24109456738 : executed forward branches (-1.9%)
          1930256656 : taken forward branches (-37.0%)
          5766776112 : executed backward branches (+8.7%)
          3485693463 : taken backward branches (-5.7%)
           778341212 : executed unconditional branches (-27.1%)
          3574484778 : all function calls (=)
          2897719906 : indirect calls (=)
           125410746 : PLT calls (=)
        160601637149 : executed instructions (-1.1%)
         27984269255 : executed load instructions (=)
                   0 : executed store instructions (=)
                   0 : taken jump table branches (=)
                   0 : taken unknown indirect branches (=)
         30654574062 : total branches (-0.9%)
          6194291331 : taken branches (-20.8%)
         24460282731 : non-taken conditional branches (+5.8%)
          5415950119 : taken conditional branches (-19.9%)
         29876232850 : all conditional branches (=)
                   0 : linker-inserted veneer calls (=)

BOLT-INFO: Starting stub-insertion pass
BOLT-INFO: Inserted 22208 stubs in the hot area and 10316 stubs in the cold area. Shared 0 times, iterated 3 times.
Segmentation fault

[kLiHz]

I'm on building Python 3.14.0a2, with llvm-19 tools.

My CPython configure flags are:

CC=clang-19 CXX=clang++-19 LD=lld-19 ./configure --enable-optimizations --prefix="$PWD/$OUTDIR/" --enable-experimental-jit --enable-bolt

Then it went Segmentation Fault like this:

Log 1

BOLT-WARNING: '-reorder-functions=hfsort+' is deprecated, please use '-reorder-functions=cdsort' instead
BOLT-INFO: Target architecture: x86_64
BOLT-INFO: BOLT version: <unknown>
BOLT-INFO: first alloc address is 0x400000
BOLT-INFO: enabling relocation mode
BOLT-INFO: enabling lite mode
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: 4917 out of 8242 functions in the binary (59.7%) have non-empty execution profile
BOLT-INFO: 16 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: 62765 instructions were shortened
BOLT-INFO: removed 707 empty blocks
BOLT-INFO: ICF folded 235 out of 8616 functions in 6 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 24.12 KB of code space. Folded functions were called 98161830 times based on profile.
BOLT-INFO: ICP Total indirect calls = 2092394554, 228 callsites cover 99% of all indirect calls
BOLT-INFO: ICP total indirect callsites with profile = 417
BOLT-INFO: ICP total jump table callsites = 68
BOLT-INFO: ICP total number of calls = 1589023449
BOLT-INFO: ICP percentage of calls that are indirect = 62.4%
BOLT-INFO: ICP percentage of indirect calls that can be optimized = 38.6%
BOLT-INFO: ICP percentage of indirect callsites that are optimized = 29.7%
BOLT-INFO: ICP number of method load elimination candidates = 0
BOLT-INFO: ICP percentage of method calls candidates that have loads eliminated = 0.0%
BOLT-INFO: ICP percentage of indirect branches that are optimized = 3.2%
BOLT-INFO: ICP percentage of jump table callsites that are optimized = 29.4%
BOLT-INFO: ICP number of jump table callsites that can use hot indices = 0
BOLT-INFO: ICP percentage of jump table callsites that use hot indices = 0.0%
BOLT-INFO: inlined 402776018 calls at 6624 call sites in 3 iteration(s). Change in binary size: 292956 bytes.
BOLT-INFO: ICF folded 4 out of 8381 functions in 3 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 0.55 KB of code space. Folded functions were called 27802861 times based on profile.
BOLT-INFO: basic block reordering modified layout of 2882 functions (58.61% of profiled, 34.40% of total)
BOLT-INFO: UCE removed 56 blocks and 0 bytes of code
BOLT-INFO: splitting separates 1585224 hot bytes from 863427 cold bytes (64.74% of split functions is hot).
BOLT-INFO: 35 Functions were reordered by LoopInversionPass
BOLT-INFO: program-wide dynostats after all optimizations before SCTC and FOP:

         15738993269 : executed forward branches
          2168360801 : taken forward branches
          4125812192 : executed backward branches
          2377275270 : taken backward branches
          2685569629 : executed unconditional branches
          2834066232 : all function calls
          1202140317 : indirect calls
           119117303 : PLT calls
        140913142253 : executed instructions
         34814353170 : executed load instructions
         18442262783 : executed store instructions
          1090286920 : taken jump table branches
                   0 : taken unknown indirect branches
         22550375090 : total branches
          7231205700 : taken branches
         15319169390 : non-taken conditional branches
          4545636071 : taken conditional branches
         19864805461 : all conditional branches

         16782277223 : executed forward branches (+6.6%)
          1565566488 : taken forward branches (-27.8%)
          3617553166 : executed backward branches (-12.3%)
          2032073552 : taken backward branches (-14.5%)
          1729122730 : executed unconditional branches (-35.6%)
          2440167425 : all function calls (-13.9%)
           792130567 : indirect calls (-34.1%)
           119117269 : PLT calls (-0.0%)
        139565804555 : executed instructions (-1.0%)
         34757674343 : executed load instructions (-0.2%)
         18436550273 : executed store instructions (-0.0%)
          1055903884 : taken jump table branches (-3.2%)
                   0 : taken unknown indirect branches (=)
         22128953119 : total branches (-1.9%)
          5326762770 : taken branches (-26.3%)
         16802190349 : non-taken conditional branches (+9.7%)
          3597640040 : taken conditional branches (-20.9%)
         20399830389 : all conditional branches (+2.7%)

BOLT-INFO: SCTC: patched 169 tail calls (168 forward) tail calls (1 backward) from a total of 169 while removing 100 double jumps and removing 209 basic blocks totalling 928 bytes of code. CTCs total execution count is 53174839 and the number of times CTCs are taken is 51967694
BOLT-INFO: FOP optimized 10 redundant load(s) and 0 unused store(s)
BOLT-INFO: Frequency of redundant loads is 16806323 and frequency of unused stores is 0
BOLT-INFO: Frequency of loads changed to use a register is 16806323 and frequency of loads changed to use an immediate is 0
BOLT-INFO: FOP deleted 7 load(s) (dyn count: 16805679) and 0 store(s)
BOLT-INFO: FRAME ANALYSIS: 3699 function(s) were not optimized.
BOLT-INFO: FRAME ANALYSIS: 1264 function(s) (55.8% dyn cov) could not have its frame indices restored.
BOLT-INFO: Shrink wrapping moved 125 spills inserting load/stores and 0 spills inserting push/pops
BOLT-INFO: Shrink wrapping reduced 1161782874 store executions (0.8% total instructions executed, 6.3% store instructions)
BOLT-INFO: Shrink wrapping failed at reducing 0 store executions (0.0% total instructions executed, 0.0% store instructions)
BOLT-INFO: Allocation combiner: 137 empty spaces coalesced (dyn count: 1407581849).
 #0 0x0000557c2263a25a (/usr/bin/llvm-bolt+0x1ccf25a)
 #1 0x0000557c22637fc6 (/usr/bin/llvm-bolt+0x1cccfc6)
 #2 0x0000557c2263aa3b (/usr/bin/llvm-bolt+0x1ccfa3b)
 #3 0x00007f334d849d20 (/lib/x86_64-linux-gnu/libc.so.6+0x3fd20)
 #4 0x0000557c22cc7320 (/usr/bin/llvm-bolt+0x235c320)
 #5 0x0000557c22cd6b8b (/usr/bin/llvm-bolt+0x236bb8b)
 #6 0x0000557c22cd78f5 (/usr/bin/llvm-bolt+0x236c8f5)
 #7 0x0000557c22cd7231 (/usr/bin/llvm-bolt+0x236c231)
 #8 0x0000557c22cd5a6a (/usr/bin/llvm-bolt+0x236aa6a)
 #9 0x0000557c226a656d (/usr/bin/llvm-bolt+0x1d3b56d)
#10 0x0000557c2269d739 (/usr/bin/llvm-bolt+0x1d32739)
#11 0x0000557c210b444f (/usr/bin/llvm-bolt+0x74944f)
#12 0x00007f334d833d68 (/lib/x86_64-linux-gnu/libc.so.6+0x29d68)
#13 0x00007f334d833e25 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e25)
#14 0x0000557c210b2351 (/usr/bin/llvm-bolt+0x747351)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /usr/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -inline-all -split-eh -reorder-functions-use-hot-size -peepholes=none -jump-tables=aggressive -inline-ap -indirect-call-promotion=all -dyno-stats -use-gnu-stack -frame-opt=hot
Segmentation fault

And I tried removing the flags mentioned in earlier replies one by one, then adding them back again, and found that it works for my by just removing --inline-all.

Log 2

/usr/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1  -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -use-gnu-stack -frame-opt=hot -jump-tables=aggressive -indirect-call-promotion=all -inline-ap
BOLT-WARNING: '-reorder-functions=hfsort+' is deprecated, please use '-reorder-functions=cdsort' instead
BOLT-INFO: Target architecture: x86_64
BOLT-INFO: BOLT version: <unknown>
BOLT-INFO: first alloc address is 0x400000
BOLT-INFO: enabling relocation mode
BOLT-INFO: enabling lite mode
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: 4917 out of 8242 functions in the binary (59.7%) have non-empty execution profile
BOLT-INFO: 16 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: 62765 instructions were shortened
BOLT-INFO: removed 707 empty blocks
BOLT-INFO: ICF folded 235 out of 8616 functions in 6 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 24.12 KB of code space. Folded functions were called 98161830 times based on profile.
BOLT-INFO: ICP Total indirect calls = 2092394554, 228 callsites cover 99% of all indirect calls
BOLT-INFO: ICP total indirect callsites with profile = 417
BOLT-INFO: ICP total jump table callsites = 68
BOLT-INFO: ICP total number of calls = 1589023449
BOLT-INFO: ICP percentage of calls that are indirect = 62.4%
BOLT-INFO: ICP percentage of indirect calls that can be optimized = 38.6%
BOLT-INFO: ICP percentage of indirect callsites that are optimized = 29.7%
BOLT-INFO: ICP number of method load elimination candidates = 0
BOLT-INFO: ICP percentage of method calls candidates that have loads eliminated = 0.0%
BOLT-INFO: ICP percentage of indirect branches that are optimized = 3.2%
BOLT-INFO: ICP percentage of jump table callsites that are optimized = 29.4%
BOLT-INFO: ICP number of jump table callsites that can use hot indices = 0
BOLT-INFO: ICP percentage of jump table callsites that use hot indices = 0.0%
BOLT-INFO: basic block reordering modified layout of 2548 functions (51.82% of profiled, 30.40% of total)
BOLT-INFO: splitting separates 1418451 hot bytes from 685362 cold bytes (67.42% of split functions is hot).
BOLT-INFO: 26 Functions were reordered by LoopInversionPass
BOLT-INFO: program-wide dynostats after all optimizations before SCTC and FOP:

         15738993269 : executed forward branches
          2168360801 : taken forward branches
          4125812192 : executed backward branches
          2377275270 : taken backward branches
          2685569629 : executed unconditional branches
          2834066232 : all function calls
          1202140317 : indirect calls
           119117303 : PLT calls
        140913142253 : executed instructions
         34814353170 : executed load instructions
         18442262783 : executed store instructions
          1090286920 : taken jump table branches
                   0 : taken unknown indirect branches
         22550375090 : total branches
          7231205700 : taken branches
         15319169390 : non-taken conditional branches
          4545636071 : taken conditional branches
         19864805461 : all conditional branches

         16748825820 : executed forward branches (+6.4%)
          1564428069 : taken forward branches (-27.9%)
          3663795535 : executed backward branches (-11.2%)
          2034833123 : taken backward branches (-14.4%)
          1712657172 : executed unconditional branches (-36.2%)
          2834066232 : all function calls (=)
           792138703 : indirect calls (-34.1%)
           119117303 : PLT calls (=)
        140256855747 : executed instructions (-0.5%)
         34790222521 : executed load instructions (-0.1%)
         18442262783 : executed store instructions (=)
          1055903884 : taken jump table branches (-3.2%)
                   0 : taken unknown indirect branches (=)
         22125278527 : total branches (-1.9%)
          5311918364 : taken branches (-26.5%)
         16813360163 : non-taken conditional branches (+9.8%)
          3599261192 : taken conditional branches (-20.8%)
         20412621355 : all conditional branches (+2.8%)

BOLT-INFO: SCTC: patched 120 tail calls (120 forward) tail calls (0 backward) from a total of 120 while removing 1 double jumps and removing 121 basic blocks totalling 605 bytes of code. CTCs total execution count is 142694592 and the number of times CTCs are taken is 133156435
BOLT-INFO: FOP optimized 10 redundant load(s) and 0 unused store(s)
BOLT-INFO: Frequency of redundant loads is 16806323 and frequency of unused stores is 0
BOLT-INFO: Frequency of loads changed to use a register is 16806323 and frequency of loads changed to use an immediate is 0
BOLT-INFO: FOP deleted 7 load(s) (dyn count: 16805679) and 0 store(s)
BOLT-INFO: FRAME ANALYSIS: 3699 function(s) were not optimized.
BOLT-INFO: FRAME ANALYSIS: 1264 function(s) (55.8% dyn cov) could not have its frame indices restored.
BOLT-INFO: Shrink wrapping moved 125 spills inserting load/stores and 1 spills inserting push/pops
BOLT-INFO: Shrink wrapping reduced 1163126871 store executions (0.8% total instructions executed, 6.3% store instructions)
BOLT-INFO: Shrink wrapping failed at reducing 0 store executions (0.0% total instructions executed, 0.0% store instructions)
BOLT-INFO: Allocation combiner: 137 empty spaces coalesced (dyn count: 1407581849).
BOLT-INFO: patched build-id (flipped last bit)
BOLT-INFO: setting _end to 0xd1122c
BOLT-INFO: setting _end to 0xd1122c
BOLT-INFO: setting __hot_start to 0xa00000
BOLT-INFO: setting __hot_end to 0xb7d557

[paschalis-mpeis]

Just pointing that the issue of @kLiHz is not on AArch64, whereas the previous issues of @belm0 and @iovxw were.

[liusy58]

Hi, got same error on python 3.13:

log

In this case, Indirect Call Promotion, and probably Inliner could have crashed later on.
For a start, can you be even more conservative, remove all the below, and optionally report whether you were able to generate a BOLTed binary?"

I can confirm it can be fixed by remove -indirect-call-promotion=all -inline-all and -frame-opt=hot

$ /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -jump-tables=aggressive -inline-ap -use-gnu-stack

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 5 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
BOLT-INFO: basic block reordering modified layout of 2372 functions (55.79% of profiled, 30.04% of total)
BOLT-INFO: 578 Functions were reordered by LoopInversionPass
BOLT-INFO: hfsort+ reduced the number of chains from 4197 to 2213
BOLT-INFO: program-wide dynostats after all optimizations before SCTC and FOP:

         24573349557 : executed forward branches
          3063021853 : taken forward branches
          5302883293 : executed backward branches
          3694579499 : taken backward branches
          1067304819 : executed unconditional branches
          3574484778 : all function calls
          2897719906 : indirect calls
           125410746 : PLT calls
        162459659259 : executed instructions
         27984269255 : executed load instructions
                   0 : executed store instructions
                   0 : taken jump table branches
                   0 : taken unknown indirect branches
         30943537669 : total branches
          7824906171 : taken branches
         23118631498 : non-taken conditional branches
          6757601352 : taken conditional branches
         29876232850 : all conditional branches
                   0 : linker-inserted veneer calls

         24109456738 : executed forward branches (-1.9%)
          1930256656 : taken forward branches (-37.0%)
          5766776112 : executed backward branches (+8.7%)
          3485693463 : taken backward branches (-5.7%)
           778341212 : executed unconditional branches (-27.1%)
          3574484778 : all function calls (=)
          2897719906 : indirect calls (=)
           125410746 : PLT calls (=)
        160601637149 : executed instructions (-1.1%)
         27984269255 : executed load instructions (=)
                   0 : executed store instructions (=)
                   0 : taken jump table branches (=)
                   0 : taken unknown indirect branches (=)
         30654574062 : total branches (-0.9%)
          6194291331 : taken branches (-20.8%)
         24460282731 : non-taken conditional branches (+5.8%)
          5415950119 : taken conditional branches (-19.9%)
         29876232850 : all conditional branches (=)
                   0 : linker-inserted veneer calls (=)

BOLT-INFO: Starting stub-insertion pass
BOLT-INFO: Inserted 22208 stubs in the hot area and 10316 stubs in the cold area. Shared 0 times, iterated 3 times.
BOLT-INFO: padding code to 0xe00000 to accommodate hot text
BOLT-INFO: setting __hot_start to 0xa00000
BOLT-INFO: setting __hot_end to 0xc0cce8

And the error log of with -inline-all:

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: creating new program header table at address 0xa00000, offset 0x800000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 4 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
 #0 0x0000aaaae0ac8c20 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1608c20)
 #1 0x0000aaaae0ac6dc0 llvm::sys::RunSignalHandlers() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1606dc0)
 #2 0x0000aaaae0ac9478 SignalHandler(int) Signals.cpp:0:0
 #3 0x0000ffff959d67bc (linux-vdso.so.1+0x7bc)
 #4 0x0000aaaae1004a94 (anonymous namespace)::AArch64MCPlusBuilder::replaceImmWithSymbolRef(llvm::MCInst&, llvm::MCSymbol const*, long, llvm::MCContext*, long&, unsigned long) const AArch64MCPlusBuilder.cpp:0:0
 #5 0x0000aaaae0f22b88 llvm::bolt::getInliningInfo(llvm::bolt::BinaryFunction const&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a62b88)
 #6 0x0000aaaae0f22ed8 llvm::bolt::Inliner::findInliningCandidates(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a62ed8)
 #7 0x0000aaaae0f24a44 llvm::bolt::Inliner::runOnFunctions(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1a64a44)
 #8 0x0000aaaae0b55d44 llvm::bolt::BinaryFunctionPassManager::runPasses() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1695d44)
 #9 0x0000aaaae0b59914 llvm::bolt::BinaryFunctionPassManager::runAllPasses(llvm::bolt::BinaryContext&) (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1699914)
#10 0x0000aaaae0b09410 llvm::bolt::RewriteInstance::run() (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x1649410)
#11 0x0000aaaadf7e18d0 main (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x3218d0)
#12 0x0000ffff953f7740 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#13 0x0000ffff953f7818 call_init ./csu/../csu/libc-start.c:128:20
#14 0x0000ffff953f7818 __libc_start_main ./csu/../csu/libc-start.c:347:5
#15 0x0000aaaadf7dfd68 _start (/mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt+0x31fd68)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -inline-all
Segmentation fault

-frame-opt=hot got segfault with no stack trace:

$ /mnt/aarch64/clang+llvm-18.1.8-aarch64-linux-gnu/bin/llvm-bolt python.prebolt -o python.bolt -data=python.fdata -update-debug-sections -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions -icf=1 -split-eh -reorder-functions-use-hot-size -peepholes=none -dyno-stats -frame-opt=hot

BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: e5def089cd9f5aa71524f82fef301ca66eaa38d2
BOLT-INFO: first alloc address is 0x200000
BOLT-INFO: creating new program header table at address 0xa00000, offset 0x800000
BOLT-INFO: enabling relocation mode
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: pre-processing profile using branch profile reader
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 4252 out of 7983 functions in the binary (53.3%) have non-empty execution profile
BOLT-INFO: 148 functions with profile could not be optimized
BOLT-INFO: profile for 1 objects was ignored
BOLT-INFO: removed 327 empty blocks
BOLT-INFO: ICF folded 461 out of 8356 functions in 6 passes. 0 functions had jump tables.
BOLT-INFO: Removing all identical functions will save 43.14 KB of code space. Folded functions were called 311268858 times based on profile.
BOLT-INFO: basic block reordering modified layout of 2372 functions (55.79% of profiled, 30.04% of total)
BOLT-INFO: 578 Functions were reordered by LoopInversionPass
BOLT-INFO: hfsort+ reduced the number of chains from 4197 to 2213
BOLT-INFO: program-wide dynostats after all optimizations before SCTC and FOP:

         24573349557 : executed forward branches
          3063021853 : taken forward branches
          5302883293 : executed backward branches
          3694579499 : taken backward branches
          1067304819 : executed unconditional branches
          3574484778 : all function calls
          2897719906 : indirect calls
           125410746 : PLT calls
        162459659259 : executed instructions
         27984269255 : executed load instructions
                   0 : executed store instructions
                   0 : taken jump table branches
                   0 : taken unknown indirect branches
         30943537669 : total branches
          7824906171 : taken branches
         23118631498 : non-taken conditional branches
          6757601352 : taken conditional branches
         29876232850 : all conditional branches
                   0 : linker-inserted veneer calls

         24109456738 : executed forward branches (-1.9%)
          1930256656 : taken forward branches (-37.0%)
          5766776112 : executed backward branches (+8.7%)
          3485693463 : taken backward branches (-5.7%)
           778341212 : executed unconditional branches (-27.1%)
          3574484778 : all function calls (=)
          2897719906 : indirect calls (=)
           125410746 : PLT calls (=)
        160601637149 : executed instructions (-1.1%)
         27984269255 : executed load instructions (=)
                   0 : executed store instructions (=)
                   0 : taken jump table branches (=)
                   0 : taken unknown indirect branches (=)
         30654574062 : total branches (-0.9%)
          6194291331 : taken branches (-20.8%)
         24460282731 : non-taken conditional branches (+5.8%)
          5415950119 : taken conditional branches (-19.9%)
         29876232850 : all conditional branches (=)
                   0 : linker-inserted veneer calls (=)

BOLT-INFO: Starting stub-insertion pass
BOLT-INFO: Inserted 22208 stubs in the hot area and 10316 stubs in the cold area. Shared 0 times, iterated 3 times.
Segmentation fault

As far as I am concerned, inline is not handled in AArch64. #120187 I have opened a pr to fix this.

Could you please give me a reproducer?

[yavtuk]

Hey @belm0, can you provide reproducer how to get python.prebolt binary?