[LLD][AArch64] Occasionally LLD drops data marker for long abs thunks

[maksfb]

In some cases, LLD may drop $d (data in code marker) symbol in the middle of created absolute long thunk. E.g.:

$ llvm-objdump -d <binary>
...
0000000013cef07c <__AArch64AbsLongThunk__foo>:
    13cef07c:   58000050        ldr     x16, 13cef084 <__AArch64AbsLongThunk__foo+0x8>
    13cef080:   d61f0200        br      x16
    13cef084:   0bce33d0        .inst   0x0bce33d0 ; undefined
    13cef088:   00000000        udf     #0 
...
$ llvm-nm --special-syms -nS <binary>
...
0000000013cef07c 0000000000000010 t __AArch64AbsLongThunk_<foo>
0000000013cef08c t $x
0000000013cef08c 0000000000000010 t __AArch64AbsLongThunk_<bar>
...

The expected output for llvm-nm should include:

0000000013cef084 t $d

The bug manifests on large binaries and less than one in a thousand thunks will have a symbol missing.

[llvmbot]

@llvm/issue-subscribers-lld-elf

Author: Maksim Panchenko (maksfb)

In some cases, LLD may drop `$d` (data in code marker) symbol in the middle of created absolute long thunk. E.g.: ``` $ llvm-objdump -d <binary> ... 0000000013cef07c <__AArch64AbsLongThunk__foo>: 13cef07c: 58000050 ldr x16, 13cef084 <__AArch64AbsLongThunk__foo+0x8> 13cef080: d61f0200 br x16 13cef084: 0bce33d0 .inst 0x0bce33d0 ; undefined 13cef088: 00000000 udf #0 ... $ llvm-nm --special-syms -nS <binary> ... 0000000013cef07c 0000000000000010 t __AArch64AbsLongThunk_<foo> 0000000013cef08c t $x 0000000013cef08c 0000000000000010 t __AArch64AbsLongThunk_<bar> ... ``` The expected output for `llvm-nm` should include: ``` 0000000013cef084 t $d ``` The bug manifests on large binaries and less than one in a thousand thunks will have a symbol missing.

[smithp35]

Can I check that version of lld that is being used?

I did come across this last year and thought it would be fixed with #116975 I'd expect that to go into LLVM 20.

With that change the only thing that I can think of is that if the call to addLongMapSyms comes after the symbol table has been committed.

If it is still a problem would it be possible to get a reproducer with --reproduce? Would save time trying to reverse engineer a failure. I expect that this is impractical via github due to size limits but if you're able to contact the Arm people working on Bolt they can get it to me.

With absolute long branch veneers I assume these are coming from no-pie binaries. As a work around --pic-veneer which should work for up to 4 GiB will produce more efficient veneers that don't need mapping symbols.

[smithp35]

I can't see an obvious problem with the code at the moment. I've put my reasoning here in case a reproducer is not possible and you need to add some diagnostic code.

lld always adds a $x on Thunk creation as all thunks need this. At this point in time lld doesn't know whether the thunk will be short or long. The extra $d mapping symbol is added when we decide that a long branch thunk is needed in https://github.com/llvm/llvm-project/blob/main/lld/ELF/Thunks.cpp#L621

The getMayUseShortThunk is called from the ThunkSection::assignOffsets https://github.com/llvm/llvm-project/blob/main/lld/ELF/SyntheticSections.cpp#L4335

The ThunkSection::assignOffsets() is called on each Thunk Section once per pass https://github.com/llvm/llvm-project/blob/main/lld/ELF/Relocations.cpp#L2483

This should mean that all Thunks have getMayUseShortThunk at the end of every pass. To make sure we converge on a fixed point, thunks are not removed, and once they go long they stay long.

If I'm right then this leaves a few possibilities on what could have gone wrong:

• We've missed a ThunkSection in https://github.com/llvm/llvm-project/blob/main/lld/ELF/Relocations.cpp#L2483 , it should be possible to test that theory by changing the order of the expression in https://github.com/llvm/llvm-project/blob/main/lld/ELF/Thunks.cpp#L646 so that getMayUseShortThunk goes first.
• We've changed addresses after createThunks in https://github.com/llvm/llvm-project/blob/main/lld/ELF/Writer.cpp#L1499 in such way that we've not called createThunks to call assignOffsets() to the thunks. It should be possible to check this by writing some diagnostic code that goes after finalizeAddressDependentContent and goes over each ThunkSection and calls getMayUseShortThunk and check that there are no changes.
• The code to add extra mapping symbols in https://github.com/llvm/llvm-project/blob/main/lld/ELF/Thunks.cpp#L675 is being called but isn't working for some reason.

[maksfb]

I did come across this last year and thought it would be fixed with #116975 I'd expect that to go into LLVM 20.

This must be it, as we are on an older version of LLD. Let me verify the fix in #116975. Thank you!