Relocatable section in no_std binary

[jfgoog]

While experimenting with no_std, I came across a possible bug when using an external crate. I say "possible" because when I discussed with a couple of folks more knowledgeable than myself, they weren't sure if it's a rustc bug, a poorly chosen default, or a case of "yeah, linkers are like that"

The attached code, no_std_hello.zip, is a no_std program that depends on an external crate (ufmt, which provides formatted output without allocations). It requires an x86_64 Linux machine.

When run, it segfaults:

$ cargo run -q --target x86_64-unknown-none
Segmentation fault

This can be fixed with -C relocation-model=static:

$ RUSTFLAGS="-C relocation-model=static" cargo run -q --target x86_64-unknown-none
42

Looking at the version compiled without RUSTFLAGS (i.e. the one that segfaults), we see that it contains relocatable code, as indicated by the DYNAMIC ELF program header:

$ readelf -l -W target/x86_64-unknown-none/debug/no_std_hello

Elf file type is DYN (Position-Independent Executable file)
Entry point 0x1c6c
There are 7 program headers, starting at offset 64

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040 0x000188 0x000188 R   0x8
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x00081a 0x00081a R   0x1000
  LOAD           0x000820 0x0000000000001820 0x0000000000001820 0x001d70 0x001d70 R E 0x1000
  LOAD           0x002590 0x0000000000004590 0x0000000000004590 0x0002e0 0x0002e0 RW  0x1000
  DYNAMIC        0x002710 0x0000000000004710 0x0000000000004710 0x0000e0 0x0000e0 RW  0x8
  GNU_RELRO      0x002590 0x0000000000004590 0x0000000000004590 0x0002e0 0x000a70 R   0x1
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0

 Section to Segment mapping:
  Segment Sections...
   00
   01     .dynsym .gnu.hash .hash .dynstr .rela.dyn .rodata
   02     .text
   03     .data.rel.ro .dynamic .got
   04     .dynamic
   05     .data.rel.ro .dynamic .got
   06

And, indeed, some examination with lldb shows that it segfaults when trying to call a function in a relocatable section. The relocatable section seems to exist because it's a (non-inlined) call to a function in the external crate (ufmt), which is compiled separately and then linked to produce the final binary:

(lldb) bt
* thread #1, name = 'no_std_hello', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x00007ffff7ffbc59 no_std_hello`ufmt::impls::ixx::_$LT$impl$u20$ufmt..uDebug$u20$for$u20$i32$GT$::fmt::h8e97f3b1fc52378d(self=0x00007ffff7ffa5bc, f=0x00007fffffffd5b8) at ixx.rs:94:21
    frame #2: 0x00007ffff7ffbbe7 no_std_hello`no_std_hello::main::_$u7b$$u7b$closure$u7d$$u7d$::h6aa378adeec3265e [inlined] ufmt::impls::ixx::_$LT$impl$u20$ufmt..uDisplay$u20$for$u20$i32$GT$::fmt::hd4a7d095957703c9(self=0x00007ffff7ffa5bc, f=0x00007fffffffd5b8) at ixx.rs:115:9
    frame #3: 0x00007ffff7ffbbe2 no_std_hello`no_std_hello::main::_$u7b$$u7b$closure$u7d$$u7d$::h6aa378adeec3265e(f=0x00007fffffffd5b8) at main.rs:85:5
    frame #4: 0x00007ffff7ffbbac no_std_hello`_$LT$W$u20$as$u20$ufmt..UnstableDoAsFormatter$GT$::do_as_formatter::haa451b76a7c159d9(self=0x00007fffffffd668, f={closure_env#0} @ 0x00007fffffffd5d0) at lib.rs:417:9
    frame #5: 0x00007ffff7ffbf85 no_std_hello`main(_stack_top="\U00000001") at main.rs:85:5
    frame #6: 0x00007ffff7ffbc74 no_std_hello`_start + 8

The relevant section from objdump -d target/x86_64-unknown-none/debug/no_std_hello shows that we are indeed calling a relocatable function: (I also confirmed in lldb that the call to this function is what's causing the crash, not something else)

0000000000001c10 <_ZN4ufmt5impls3ixx46_$LT$impl$u20$ufmt..uDebug$u20$for$u20$i32$GT$3fmt17h8e97f3b1fc52378dE>:
    1c10:       48 83 ec 38             sub    $0x38,%rsp
    1c14:       48 89 74 24 08          mov    %rsi,0x8(%rsp)
    1c19:       48 89 7c 24 20          mov    %rdi,0x20(%rsp)
    1c1e:       48 89 74 24 28          mov    %rsi,0x28(%rsp)
    1c23:       8a 4c 24 37             mov    0x37(%rsp),%cl
    1c27:       48 8d 44 24 15          lea    0x15(%rsp),%rax
    1c2c:       0f b6 c9                movzbl %cl,%ecx
    1c2f:       48 ba 01 01 01 01 01    movabs $0x101010101010101,%rdx
    1c36:       01 01 01
    1c39:       48 0f af ca             imul   %rdx,%rcx
    1c3d:       48 89 08                mov    %rcx,(%rax)
    1c40:       89 48 07                mov    %ecx,0x7(%rax)
    1c43:       48 63 3f                movslq (%rdi),%rdi
    1c46:       48 8b 05 d3 2b 00 00    mov    0x2bd3(%rip),%rax        # 4820 <_DYNAMIC+0x110>
    1c4d:       48 8d 74 24 15          lea    0x15(%rsp),%rsi
    1c52:       ba 0b 00 00 00          mov    $0xb,%edx
    1c57:       ff d0                   call   *%rax
    1c59:       48 8b 7c 24 08          mov    0x8(%rsp),%rdi
    1c5e:       48 89 c6                mov    %rax,%rsi
    1c61:       e8 ca 03 00 00          call   2030 <_ZN4ufmt18Formatter$LT$W$GT$9write_str17h70e7e896fe1e8040E>
    1c66:       48 83 c4 38             add    $0x38,%rsp
    1c6a:       c3                      ret
    1c6b:       cc                      int3

My (rudimentary) understanding of ELF files is that when you have relocatable code (either because of a separate shared library or, as in this case, in the program itself), you would expect to see an INTERP header that invokes ld.so to fix up the relocatable addresses. In this case, however, with --target x86_64-unknown-none, it would be wrong (not to mention impossible) to construct such a header because we have no idea what the operating system is.

So...is this a bug? Or does no_std come with an expectation that you need to understand and customize how your program is linked? If it's not a bug, would a warning message be appropriate?

[joboet]

You're doing the equivalent of trying to run a binary compiled for macOS on Linux. This will in all likelihood not work.

x86_64-unknown-none is used for kernels or similar bare-metal software, where you have precise control over the (boot)loader and the linking it does. For simply programming for Linux without std, the default x86_64-unknown-linux-gnu or x86_64-unknown-linux-musl is still the correct target. Note that x86_64-unknown-linux-gnu will still link against glibc, because even #![no_std] programs need memcpy and the like. I don't know if musl does this as well, but if so, you can define your own target if you want. Still, the #![no_std] flag alone is enough to tell rustc not to link std, there should be no need to switch targets.

Sidenote: I find it strange that Linux does not detect that the ELF file expects a different system, as the OSABI byte in the ELF header is a different value for x86_64-unknown-none (SYSV), but that behaviour is highly unlikely to change, because "kernel will never break userspace" and someone will rely on it.

[jfgoog]

Thanks for the feedback.

Running the program compiled for x86_64-unknown-none does work, and behaves as expected when linked statically. But, you do have to provide some OS- and ABI-specific code that would normally be provided by libc, such as a _start symbol written in assembly, some code to understand what's on the stack, and some way to make syscalls.

Also, I think memcpy is special in that even though it's ostensibly part of libc, the implementation is typically supplied by the compiler. In fact, when I do objdump -d on my program, I see a function called _ZN17compiler_builtins3mem6memcpy17h058b7d6261135f8fE

[joboet]

Statically linking just happens to work because Linux uses very similar binaries to what x86_64-unknown-none produces. But fundamentally, x86_64-unknown-none is not designed for Linux. That's why, because dynamic linking is enabled by default for that target, the binary crashes. Cargo doesn't warn you if you try to cargo run a binary that's compiled for a different target than the host target because sometimes, that actually works (think 32-bit programs on 64-bit Linux or x86_64 programs on aarch64 macOS under Rosetta).

compiler_builtins is linked for convenience on x86_64-unknown-none, because there is no libc to provide it. On the *-*-linux-* targets, libc is available, so it gets used (this enables some nice optimizations because the GNU dyld can select the best memcpy implementation depending on available CPU features). Also see the crate documentation for core.

This all is (sadly) completely expected and correct behaviour (well, except the Linux part I mentioned). ELF unfortunately makes it very difficult to detect which features the ELF loader needs to support in order for the binary to run correctly (which is why the kernel can't tell you "hey, this code needs relocations, but I don't do those"), but that's just the choice most UNIX platforms made.

Edit: If you're looking for a good read, fasterthanlime did a fantastic series that covers ELF loading and linking.

[jfgoog]

What you are saying seems to boil down to "Not a bug. If you are using no_std and x86_64-unknown-none you MUST understand how your program is being linked and there is no way to pick a reasonable default.". In which case, if I am understanding you correctly, feel free to close the bug.

Thanks for taking the time to respond.

[joboet]

What you are saying seems to boil down to "Not a bug. If you are using no_std and x86_64-unknown-none you MUST understand how your program is being linked and there is no way to pick a reasonable default.". In which case, if I am understanding you correctly, feel free to close the bug.

Pretty much. What you're trying to do (a bare-bones Linux binary without any dependencies) is possible by using a custom target, if you feel like going down that road.

I can't close the issue, I'm not a team member.

Thanks for taking the time to respond.

You're welcome 😄!