llvm-bolt instrument appears Segmentation fault (core dumped)

[qianwindz]

Testing a simple SO library on arm64 resulted in the following error：

The code for this library is as follows:
#include

extern "C" {

attribute((visibility("default")))
int32_t add(int32_t a, int32_t b) {
return a + b;
}

attribute((visibility("default")))
int32_t multiply(int32_t x, int32_t y) {
return x * y;
}S

} // extern "C"
The libtestdemo format is:
ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[md5/uuid]=2d020c716e8f70c88d6cf7306c2c9edd, with debug_info, not stripped
llvm-bolt version is:

the clang compiler version is:
clang-r522817

[llvmbot]

@llvm/issue-subscribers-bolt

Author: None (qianwindz)

Testing a simple SO library on arm64 resulted in the following error：  The code for this library is as follows: #include <cstdint>

extern "C" {

attribute((visibility("default")))
int32_t add(int32_t a, int32_t b) {
return a + b;
}

attribute((visibility("default")))
int32_t multiply(int32_t x, int32_t y) {
return x * y;
}S

} // extern "C"
The libtestdemo format is:
ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[md5/uuid]=2d020c716e8f70c88d6cf7306c2c9edd, with debug_info, not stripped

[maksfb]

Most likely, it's the same issue as #136669. You can verify by running readelf -d <binary> and checking for INIT entry.

[qianwindz]

@maksfb hi,I use the readelf -d and the result is:
Dynamic section at offset 0x8018 contains 25 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libc++.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so]
0x0000000000000001 (NEEDED) Shared library: [libm.so]
0x0000000000000001 (NEEDED) Shared library: [libdl.so]
0x000000000000000e (SONAME) Library soname: [libtestdemo.so]
0x000000000000001e (FLAGS) BIND_NOW
0x000000006ffffffb (FLAGS_1) Flags: NOW
0x0000000000000024 (RELR) 0x388
0x0000000000000023 (RELRSZ) 16 (bytes)
0x0000000000000025 (RELRENT) 8 (bytes)
0x0000000000000017 (JMPREL) 0x398
0x0000000000000002 (PLTRELSZ) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x81a8
0x0000000000000014 (PLTREL) RELA
0x0000000000000006 (SYMTAB) 0x288
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000005 (STRTAB) 0x334
0x000000000000000a (STRSZ) 84 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x310
0x000000000000001a (FINI_ARRAY) 0x8008
0x000000000000001c (FINI_ARRAYSZ) 16 (bytes)
0x000000006ffffff0 (VERSYM) 0x2e8
0x000000006ffffffe (VERNEED) 0x2f0
0x000000006fffffff (VERNEEDNUM) 1
0x0000000000000000 (NULL) 0x0
readelf: Warning: Unrecognized form: 0x23
readelf: Warning: Unrecognized form: 0x22
readelf: Warning: Unrecognized form: 0x23
readelf: Warning: Unrecognized form: 0x22
readelf: Warning: Unrecognized form: 0x22
I don't know if this result is correct

[qianwindz]

I also tried bolt optimization on another so library, but this library encountered the following error:

BOLT-INFO: 1164 out of 3035 CUs will be updated
BOLT-INFO: potentially escaped address 0x297adc in function _ZN7android10uirenderer19VectorDrawableUtils11verbsToPathEP6SkPathRKNS0_14VectorDrawable4Path4DataE/1(*2)
BOLT-INFO: potentially escaped address 0x297b90 in function _ZN7android10uirenderer19VectorDrawableUtils11verbsToPathEP6SkPathRKNS0_14VectorDrawable4Path4DataE/1(*2)
BOLT-INFO: potentially escaped address 0x2a2868 in function _ZN7android10uirenderer10PathParser21validateVerbAndPointsEcmPNS1_11ParseResultE/1(*2)
BOLT-INFO: potentially escaped address 0x2f5430 in function skcms_Transform/1(*2)
BOLT-INFO: potentially escaped address 0x2f5df4 in function skcms_Transform/1(*2)
BOLT-INFO: potentially escaped address 0x301720 in function _ZN10SkBmpCodec10ReadHeaderEP8SkStreambPNSt3__110unique_ptrI7SkCodecNS2_14default_deleteIS4_EEEE/1(*2)
BOLT-INFO: potentially escaped address 0x32f298 in function _Z24SkBlendMode_AppendStages11SkBlendModeP16SkRasterPipeline/1(*2)
BOLT-INFO: potentially escaped address 0x32f7a4 in function _ZN9SkBlender4ModeE11SkBlendMode/1(*2)
BOLT-INFO: potentially escaped address 0x3548fc in function _Z15SkConvertPixelsRK11SkImageInfoPvmS1_PKvm/1(*2)
BOLT-INFO: potentially escaped address 0x37b5f8 in function _ZNK11SkImageInfo15computeByteSizeEm
BOLT-INFO: potentially escaped address 0x37cf58 in function Z28SkColorTypeValidateAlphaType11SkColorType11SkAlphaTypePS0/1(*2)
BOLT-INFO: potentially escaped address 0x37cfe0 in function _ZN21SkKnownRuntimeEffects21GetKnownRuntimeEffectENS_9StableKeyE/1(*2)
BOLT-INFO: potentially escaped address 0x391a50 in function _ZN8SkMipmap15MakeDownSamplerERK8SkPixmap/1(*2)
BOLT-INFO: potentially escaped address 0x3a22e8 in function ZN4neonL7shuffleEP21SkRasterPipelineStagemmPSt4byteDv4_fS4_S4_S4_S4_S4_S4_S4.__uniq.171973291814702829445853707239222793327/1(*2)
BOLT-INFO: potentially escaped address 0x3be5fc in function _ZN17SkPicturePlayback4drawEP8SkCanvasPN9SkPicture13AbortCallbackEP12SkReadBuffer/1(*2)
BOLT-INFO: potentially escaped address 0x3c6444 in function _ZNK8SkPixmap8getColorEii/1(*2)
BOLT-INFO: potentially escaped address 0x3c6d00 in function _ZNK8SkPixmap15computeIsOpaqueEv/1(*2)
BOLT-INFO: potentially escaped address 0x3c77cc in function _ZNK8SkPixmap5eraseERK8SkRGBA4fIL11SkAlphaType3EEPK7SkIRect/1(*2)
BOLT-INFO: potentially escaped address 0x3cbf08 in function _ZN16SkRasterPipeline15uncheckedAppendE18SkRasterPipelineOpPv/1(*2)
BOLT-INFO: potentially escaped address 0x3ccdc8 in function _ZN16SkRasterPipeline10appendLoadE11SkColorTypePK26SkRasterPipeline_MemoryCtx/1(*2)
BOLT-INFO: potentially escaped address 0x3cd3a8 in function _ZN16SkRasterPipeline13appendLoadDstE11SkColorTypePK26SkRasterPipeline_MemoryCtx/1(*2)
BOLT-INFO: potentially escaped address 0x3cd7d0 in function _ZN16SkRasterPipeline11appendStoreE11SkColorTypePK26SkRasterPipeline_MemoryCtx/1(*2)
BOLT-INFO: potentially escaped address 0x3d3224 in function ZN8SkRecord6Record6mutateIRNS_9DestroyerEEEDTclfp_cvPN9SkRecords4NoOpELDnEEEOT/1(*2)
BOLT-INFO: potentially escaped address 0x3d39a0 in function _Z18SkRecordFillBoundsRK6SkRectRK8SkRecordPS_PN15SkBBoxHierarchy8MetadataE/1(*2)
BOLT-INFO: potentially escaped address 0x3d4494 in function ZNK8SkRecord6Record5visitIRN9SkRecords4DrawEEEDTclfp_cvNS2_4NoOpE_EEEOT/1(*2)
BOLT-INFO: potentially escaped address 0x408a8c in function _ZN10SkYUVAInfo16GetYUVALocationsENS_11PlaneConfigEPKj/1(*2)
BOLT-INFO: potentially escaped address 0x42b458 in function _ZN15SkPngEncoderMgr10chooseProcERK11SkImageInfo/1(*2)
BOLT-INFO: potentially escaped address 0x47fd40 in function _ZNK10SkPDFUnion10emitObjectEP9SkWStream/1(*2)
BOLT-INFO: potentially escaped address 0x48eb80 in function _ZN12_GLOBAL__N_121colrv1_traverse_paintEP8SkCanvasRK6SkSpanIjEjP11FT_FaceRec_16FT_Opaque_Paint_PN12skia_private8THashSetIS8_NS_17OpaquePaintHasherEEE.__uniq.86317533073000855509244020988992072466/1(*2)
BOLT-INFO: potentially escaped address 0x492724 in function _ZN12_GLOBAL__N_128colrv1_traverse_paint_boundsEP8SkMatrixP6SkRectP11FT_FaceRec_16FT_Opaque_Paint_PN12skia_private8THashSetIS6_NS_17OpaquePaintHasherEEE.__uniq.86317533073000855509244020988992072466/1(*2)
BOLT-INFO: potentially escaped address 0x497240 in function _ZN15SkGainmapShader4MakeERK5sk_spIK7SkImageERK6SkRectRK17SkSamplingOptionsS5_S8_SB_RK13SkGainmapInfoS8_fS0_I12SkColorSpaceE/1(*2)
BOLT-INFO: potentially escaped address 0x49a410 in function _ZZNK13SkImageShader12appendStagesERK10SkStageRecRKN9SkShaders9MatrixRecEENK3$_3clEPKN12_GLOBAL__N_114MipLevelHelperE.__uniq.311361382650976851747884884028897267701/1(2)
BOLT-INFO: potentially escaped address 0x4a42d4 in function _ZN4SkSL15TProgramVisitorINS_19ProgramVisitorTypesEE15visitExpressionERKNS_10ExpressionE/1(2)
BOLT-INFO: potentially escaped address 0x4a47b4 in function _ZN4SkSL15TProgramVisitorINS_18ProgramWriterTypesEE15visitExpressionERNS_10ExpressionE/1(2)
#0 0x000055aa9e2146bf llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/vboxuser/Desktop/llvm-project/llvm/lib/Support/Unix/Signals.inc:804:22
#1 0x000055aa9e211ec4 llvm::sys::RunSignalHandlers() /home/vboxuser/Desktop/llvm-project/llvm/lib/Support/Signals.cpp:105:20
#2 0x000055aa9e211ec4 SignalHandler(int, siginfo_t, void) /home/vboxuser/Desktop/llvm-project/llvm/lib/Support/Unix/Signals.inc:405:31
#3 0x00007f2cb2251520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
#4 0x000055aa9ea6ea36 llvm::MCSymbol::getName() const /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/MC/MCSymbol.h:206:5
#5 0x000055aa9ea6ea36 llvm::bolt::BinaryContext::getSymbolValue(llvm::MCSymbol const&) const /home/vboxuser/Desktop/llvm-project/bolt/include/bolt/Core/BinaryContext.h:926:47
#6 0x000055aa9ea6ea36 llvm::bolt::AArch64MCSymbolizer::adjustRelocation(llvm::bolt::Relocation const&, llvm::MCInst const&) const /home/vboxuser/Desktop/llvm-project/bolt/lib/Target/AArch64/AArch64MCSymbolizer.cpp:126:66
#7 0x000055aa9ea6ee0a std::_Optional_base_impl<llvm::bolt::Relocation, std::_Optional_base<llvm::bolt::Relocation, true, true>>::_M_is_engaged() const /usr/include/c++/9/optional:432:58
#8 0x000055aa9ea6ee0a std::optionalllvm::bolt::Relocation::operator bool() const /usr/include/c++/9/optional:913:36
#9 0x000055aa9ea6ee0a llvm::bolt::AArch64MCSymbolizer::tryAddingSymbolicOperand(llvm::MCInst&, llvm::raw_ostream&, long, unsigned long, bool, unsigned long, unsigned long, unsigned long) /home/vboxuser/Desktop/llvm-project/bolt/lib/Target/AArch64/AArch64MCSymbolizer.cpp:50:20
#10 0x000055aa9f6c1ade llvm::MCDisassembler::tryAddingSymbolicOperand(llvm::MCInst&, long, unsigned long, bool, unsigned long, unsigned long, unsigned long) const /home/vboxuser/Desktop/llvm-project/llvm/lib/MC/MCDisassembler/MCDisassembler.cpp:39:1
#11 0x000055aa9db90feb DecodeAdrInstruction /home/vboxuser/Desktop/llvm-project/llvm/lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp:1514:3
#12 0x000055aa9db90feb llvm::MCDisassembler::DecodeStatus llvm::decodeToMCInst(llvm::MCDisassembler::DecodeStatus, unsigned int, unsigned int, llvm::MCInst&, unsigned long, llvm::MCDisassembler const, bool&) /home/vboxuser/Desktop/build/lib/Target/AArch64/AArch64GenDisassemblerTables.inc:36761:15
#13 0x000055aa9dba13b9 decodeInstruction /home/vboxuser/Desktop/build/lib/Target/AArch64/AArch64GenDisassemblerTables.inc:38479:7
#14 0x000055aa9dba13b9 llvm::AArch64Disassembler::getInstruction(llvm::MCInst&, unsigned long&, llvm::ArrayRef, unsigned long, llvm::raw_ostream&) const /home/vboxuser/Desktop/llvm-project/llvm/lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp:251:26
#15 0x000055aa9ead5da9 llvm::bolt::BinaryFunction::disassemble() /home/vboxuser/Desktop/llvm-project/bolt/lib/Core/BinaryFunction.cpp:1305:5
#16 0x000055aa9e2bbad5 llvm::Error::getPtr() const /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/Support/Error.h:278:51
#17 0x000055aa9e2bbad5 llvm::Error::operator=(llvm::Error&&) /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/Support/Error.h:218:11
#18 0x000055aa9e2bbad5 llvm::Error::Error(llvm::Error&&) /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/Support/Error.h:198:28
#19 0x000055aa9e2bbad5 handleAllErrors<llvm::bolt::RewriteInstance::disassembleFunctions()::<lambda(const llvm::bolt::BOLTError&)> > /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/Support/Error.h:977:11
#20 0x000055aa9e2bbad5 llvm::bolt::RewriteInstance::disassembleFunctions() /home/vboxuser/Desktop/llvm-project/bolt/lib/Rewrite/RewriteInstance.cpp:3362:20
#21 0x000055aa9e303018 llvm::bolt::RewriteInstance::run() /home/vboxuser/Desktop/llvm-project/bolt/lib/Rewrite/RewriteInstance.cpp:702:24
#22 0x000055aa9d6b378f llvm::Error::setChecked(bool) /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/Support/Error.h:309:22
#23 0x000055aa9d6b378f llvm::Error::operator bool() /home/vboxuser/Desktop/llvm-project/llvm/include/llvm/Support/Error.h:241:15
#24 0x000055aa9d6b378f main /home/vboxuser/Desktop/llvm-project/bolt/tools/driver/llvm-bolt.cpp:260:29
#25 0x00007f2cb2238d90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#26 0x00007f2cb2238e40 call_init ./csu/../csu/libc-start.c:128:20
#27 0x00007f2cb2238e40 __libc_start_main ./csu/../csu/libc-start.c:379:5
#28 0x000055aa9d75c83e _start (./llvm-bolt+0x38583e)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: ./llvm-bolt libxx.so -instrument -update-debug-sections -o ./libxx_instrument.so -v 2

and in this so library,the readelf -d result is:

0x000000006ffffffb (FLAGS_1) Flags: NOW
0x0000000060000011 (Operating System specific: 60000011) 0x122d0
0x0000000060000012 (Operating System specific: 60000012) 0xb25
0x0000000000000009 (RELAENT) 24 (bytes)
0x0000000000000024 (RELR) 0x12df8
0x0000000000000023 (RELRSZ) 2792 (bytes)
0x0000000000000025 (RELRENT) 8 (bytes)
0x0000000000000017 (JMPREL) 0x138e0
0x0000000000000002 (PLTRELSZ) 22584 (bytes)
0x0000000000000003 (PLTGOT) 0x828bf0
0x0000000000000014 (PLTREL) RELA
0x0000000000000006 (SYMTAB) 0x2f8
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000005 (STRTAB) 0x865c
0x000000000000000a (STRSZ) 40045 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x80d0
0x0000000000000019 (INIT_ARRAY) 0x8275c8
0x000000000000001b (INIT_ARRAYSZ) 208 (bytes)
0x000000000000001a (FINI_ARRAY) 0x8275b8
0x000000000000001c (FINI_ARRAYSZ) 16 (bytes)
0x000000006ffffff0 (VERSYM) 0x7540
0x000000006ffffffc (VERDEF) 0x7ec8
0x000000006ffffffd (VERDEFNUM) 3
0x000000006ffffffe (VERNEED) 0x7f1c
0x000000006fffffff (VERNEEDNUM) 9
0x0000000000000000 (NULL) 0x0
readelf: Warning: Unrecognized form: 0x23
readelf: Warning: Unrecognized form: 0x22
readelf: Warning: Unrecognized form: 0x23
readelf: Warning: Unrecognized form: 0x22
readelf: Warning: Unrecognized form: 0x22
readelf: Warning: Unrecognized form: 0x23
readelf: Warning: Unrecognized form: 0x23

[paschalis-mpeis]

In both cases the INIT entry is missing from .dynamic. We could probably make bolt exit with a helpful message on such cases.

[qianwindz]

@maksfb
I have another question about bolt:
I added void _init (void) {} in the source code section of the SO library to solve the DT-INIT issue, but the following error occurred. I found that this error has been fixed when I checked the issue, but I built it on Ubuntu 20.04 based on the latest source code. According to my understanding, this problem should not occur?Looking forward to your prompt reply

BOLT-INFO: Finished pass: long-jmp
BOLT-INFO: Starting pass: finalize-functions
BOLT-INFO: Finished pass: finalize-functions
BOLT-INFO: Starting pass: frame-optimizer
BOLT-INFO: Finished pass: frame-optimizer
BOLT-INFO: Starting pass: alloc-combiner
BOLT-INFO: Finished pass: alloc-combiner
BOLT-INFO: Starting pass: retpoline-insertion
BOLT-INFO: Finished pass: retpoline-insertion
BOLT-INFO: Starting pass: assign-sections
BOLT-INFO: Finished pass: assign-sections
BOLT-INFO: Starting pass: patch-entries
BOLT-INFO: Finished pass: patch-entries
BOLT-INFO: Starting pass: inst-lowering
BOLT-INFO: Finished pass: inst-lowering
BOLT-INFO: Starting pass: lower-annotations
BOLT-INFO: Finished pass: lower-annotations
BOLT-INFO: Starting pass: clean-mc-state
BOLT-INFO: Finished pass: clean-mc-state
BOLT-INFO: padding code to 0x600000 to accommodate hot text
BOLT-ERROR: linking object with arch x86_64 into context with arch aarch64

[qianwindz]

@paschalis-mpeis thankyou,So I can understand that both libraries were blocked by the same error？

[paschalis-mpeis]

BOLT-ERROR: linking object with arch x86_64 into context with arch aarch64

This is the same issue with #93151. This check was enforced by:

• [BOLT] Err when linking objects of different architectures #66770

You are trying to cross-compile from an x86 host a AArch64 binary?

[qianwindz]

@paschalis-mpeis
Thank you for your reply. Yes, I compiled the Android SO library on an x86_64 host. however #100866 has merged.Is there any way to solve this problem? For example, rebuilding and so on