Building core::sync::atomic for BPF target fails

[vadorovsky]

Starting from the following commit and PR, which enabled CAS for BPF target and got included in nightly:

#105708
11331b1

building the rustc library with the following config:

[build]
target = ["x86_64-unknown-linux-gnu", "bpfel-unknown-none"]

fails with:

https://gist.github.com/vadorovsky/c9a3892a85ea4425082d91a729ffcfb2

A similar failure can be observed when building an eBPF program based on https://github.com/aya-rs/aya:

https://gist.github.com/vadorovsky/7aaf1bbbb78ab0c0371fe65d90105007

That error occurs, because all these macros and functions which "can't be found", are annotated with #[cfg(target_has_atomic = "8")] or #[cfg(target_has_atomic_load_store = "8")].

Example:

rust/library/core/src/sync/atomic.rs

Lines 3014 to 3016 in bfffe40

	#[cfg(target_has_atomic = "8")]
	#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces
	unsafe fn atomic_add<T: Copy>(dst: *mut T, val: T, order: Ordering) -> T {

Those annotations prevent those macros and functions to be built for BPF, because we enable only 64-bit atomics for BPF:

rust/compiler/rustc_target/src/spec/bpf_base.rs

Lines 25 to 26 in bfffe40

	min_atomic_width: Some(64),
	max_atomic_width: Some(64),

A quick fix to make the build fix successful is changing annotations to:

#[cfg(any(
    target_has_atomic = "8",
    target_has_atomic = "16",
    target_has_atomic = "32",
    target_has_atomic = "64",
    target_has_atomic = "128",
    target_has_atomic = "ptr"
))]

I made this change on my local branch:

vadorovsky@041978b

It fixes the core library build, however, it doesn't make the most of atomic operations actually working. I made a test Aya-based project where I tried to use all atomic operations on AtomicU64:

https://github.com/vadorovsky/aya-examples/tree/main/atomic
https://github.com/vadorovsky/aya-examples/blob/main/atomic/atomic-ebpf/src/main.rs

But pretty much every atomic operation except store, swap and assigning to mutable raw pointer triggers various bpf-linker / LLVM errors like this one (when using fetch_add):

     Running `rustc --crate-name atomic --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=274 --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto -C codegen-units=1 -C metadata=0df4b88cd767cdfe -C extra-filename=-0df4b88cd767cdfe --out-dir /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps --target bpfel-unknown-none -L dependency=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps -L dependency=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/debug/deps --extern atomic_common=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/libatomic_common-9766a80d5f618b6a.rlib --extern aya_bpf=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/libaya_bpf-6a252bb26304b8e6.rlib --extern aya_log_ebpf=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/libaya_log_ebpf-5apf/../target/bpfel-unknown-none/debug/deps/libcore-88a9f41520fe0405.rlib' -Z unstable-options`
error: linking with `bpf-linker` failed: signal: 6 (SIGABRT) (core dumped)
  |
  = note: LC_ALL="C" PATH="/home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/vadorovsky/.npm-global/bin:/home/vadorovsky/solana/bin:/home/vadorovsky/.local/share/solana/install/active_release/bin:/home/vadorovsky/.local/llvm/bin:/home/vadorovsky/go/bin:/home/vadorovsky/.cargo/bin:/home/vadorovsky/miniconda3/bin:/home/vadorovsky/miniconda3/condabin:/home/vadorovsky/.cargo/bin:/home/vadorovsky/.local/bin:/home/vadorovsky/bin:/usr/local/bin:/usr/bin:/usr/bin" VSLANG="1033" "bpf-linker" "--export-symbols" "/tmp/rustcOxihKW/symbols" "/tmp/rustcOxihKW/symbols.o" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe.atomic.9f2f50cf-cgu.0.rcgu.o" "-L" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps" "-L" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/debug/deps" "-L" "/home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib" "--cpu" "generic" "--cpu-features" "" "-L" "/home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib" "-o" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe" "-O3" "--debug"
  = note: 06:08:59 [ERROR] fatal error: "Cannot select: t88: i64,ch = AtomicLoad<(dereferenceable load monotonic (s64) from %ir.7)> t1211, FrameIndex:i64<5>\n  t87: i64 = FrameIndex<5>\nIn function: atomic"
          PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
          Stack dump:
          0.    Running pass 'Function Pass Manager' on module 'atomic-0df4b88cd767cdfe'.
          1.    Running pass 'BPF DAG->DAG Pattern Instruction Selection' on function '@atomic'
          06:08:59 [ INFO] command line: "bpf-linker --export-symbols /tmp/rustcOxihKW/symbols /tmp/rustcOxihKW/symbols.o /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe.atomic.9f2f50cf-cgu.0.rcgu.o -L /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps -L /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/debug/deps -L /home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib --cpu generic --cpu-features  -L /home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib -o /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe -O3 --debug"
          06:08:59 [ INFO] LLVM command line: ["bpf-linker", "--bpf-expand-memcpy-in-order"]
          06:08:59 [ INFO] linking file "/tmp/rustcOxihKW/symbols.o" type elf
          06:08:59 [ WARN] ignoring file "/tmp/rustcOxihKW/symbols.o": no embedded bitcode
          06:08:59 [ INFO] linking file "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe.atomic.9f2f50cf-cgu.0.rcgu.o" type bitcode
          06:08:59 [ INFO] emitting LLVMObjectFile to "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe"

Or this one (when using compare_exchange):

     Running `rustc --crate-name atomic --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=274 --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto -C codegen-units=1 -C metadata=0df4b88cd767cdfe -C extra-filename=-0df4b88cd767cdfe --out-dir /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps --target bpfel-unknown-none -L dependency=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps -L dependency=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/debug/deps --extern atomic_common=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/libatomic_common-9766a80d5f618b6a.rlib --extern aya_bpf=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/libaya_bpf-6a252bb26304b8e6.rlib --extern aya_log_ebpf=/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/libaya_log_ebpf-5apf/../target/bpfel-unknown-none/debug/deps/libcore-88a9f41520fe0405.rlib' -Z unstable-options`
error: linking with `bpf-linker` failed: signal: 6 (SIGABRT) (core dumped)
  |
  = note: LC_ALL="C" PATH="/home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/vadorovsky/.npm-global/bin:/home/vadorovsky/solana/bin:/home/vadorovsky/.local/share/solana/install/active_release/bin:/home/vadorovsky/.local/llvm/bin:/home/vadorovsky/go/bin:/home/vadorovsky/.cargo/bin:/home/vadorovsky/miniconda3/bin:/home/vadorovsky/miniconda3/condabin:/home/vadorovsky/.cargo/bin:/home/vadorovsky/.local/bin:/home/vadorovsky/bin:/usr/local/bin:/usr/bin:/usr/bin" VSLANG="1033" "bpf-linker" "--export-symbols" "/tmp/rustcAlOvk2/symbols" "/tmp/rustcAlOvk2/symbols.o" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe.atomic.9f2f50cf-cgu.0.rcgu.o" "-L" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps" "-L" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/debug/deps" "-L" "/home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib" "--cpu" "generic" "--cpu-features" "" "-L" "/home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib" "-o" "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe" "-O3" "--debug"
  = note: 06:10:02 [ERROR] fatal error: "Cannot select: t88: i64,ch = AtomicLoad<(dereferenceable load monotonic (s64) from %ir.7)> t1211, FrameIndex:i64<5>\n  t87: i64 = FrameIndex<5>\nIn function: atomic"
          PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
          Stack dump:
          0.    Running pass 'Function Pass Manager' on module 'atomic-0df4b88cd767cdfe'.
          1.    Running pass 'BPF DAG->DAG Pattern Instruction Selection' on function '@atomic'
          06:10:02 [ INFO] command line: "bpf-linker --export-symbols /tmp/rustcAlOvk2/symbols /tmp/rustcAlOvk2/symbols.o /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe.atomic.9f2f50cf-cgu.0.rcgu.o -L /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps -L /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/debug/deps -L /home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib --cpu generic --cpu-features  -L /home/vadorovsky/repos/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/bpfel-unknown-none/lib -o /home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe -O3 --debug"
          06:10:02 [ INFO] LLVM command line: ["bpf-linker", "--bpf-expand-memcpy-in-order"]
          06:10:02 [ INFO] linking file "/tmp/rustcAlOvk2/symbols.o" type elf
          06:10:02 [ WARN] ignoring file "/tmp/rustcAlOvk2/symbols.o": no embedded bitcode
          06:10:02 [ INFO] linking file "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe.atomic.9f2f50cf-cgu.0.rcgu.o" type bitcode
          06:10:02 [ INFO] emitting LLVMObjectFile to "/home/vadorovsky/repos/aya-examples/atomic/atomic-ebpf/../target/bpfel-unknown-none/debug/deps/atomic-0df4b88cd767cdfe"

I didn't investigate the bpf-linker / LLVM failures deeper yet. It might be related to the fact that BPF target is calling intrinsics manually and was written only with clang in mind. But again, I need more time to investigate.

But at this point I'm quite confident that we should just revert #105708 and reconsider such change after we make sure it actually works and all the mentioned issues are fixed.

/cc @tomerze @alessandrod

[bjorn3]

Re-opening as #106856 will need to land too.