When LTO is enabled, miscompiles (x86_64-apple-darwin, at least) and failures to link due to unknown relocation type in switch tables (AArch64 Linux/Android, at least)

[briansmith]

This is from https://github.com/briansmith/ring 's CI, testing the tip of the main branch. The CI passes on 1.86 and fails on multiple targets when building with 1.87. In at least one case, building and linking passes but the test fails, indicating potentially the code (computing HMAC) or the test of the code is being miscompiled.

Unfortunately, I have limited access to systems that would help me reduce these today. I have noticed another project has run into a similar linker error recently. I apologize in advance for such a poor bug report. I hope I, or ideally others, could fill in the details, as I have limited availability today.

On aarch64-unknown-linux-gnu

https://github.com/briansmith/ring/actions/runs/15144547125/job/42577716380#logs

mk/cargo.sh +stable test -vv --target=aarch64-unknown-linux-gnu --release fails to link a test.

     Running `CARGO=/home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_CRATE_NAME=rsa_tests CARGO_MANIFEST_DIR=/home/runner/work/ring/ring CARGO_MANIFEST_PATH=/home/runner/work/ring/ring/Cargo.toml CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='An experiment.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='Apache-2.0 AND ISC' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=ring CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/briansmith/ring' CARGO_PKG_RUST_VERSION=1.66.0 CARGO_PKG_VERSION=0.17.14 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=17 CARGO_PKG_VERSION_PATCH=14 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 CARGO_SBOM_PATH='' CARGO_TARGET_TMPDIR=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/tmp LD_LIBRARY_PATH='/home/runner/work/ring/ring/target/release/deps:/home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' OUT_DIR=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/build/ring-9ec23736496dbc30/out /home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name rsa_tests --edition=2021 tests/rsa_tests.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --emit=dep-info,link -C opt-level=3 -C lto -C codegen-units=1 -C overflow-checks=on --test --cfg 'feature="alloc"' --cfg 'feature="default"' --cfg 'feature="dev_urandom_fallback"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("alloc", "default", "dev_urandom_fallback", "less-safe-getrandom-custom-or-rdrand", "less-safe-getrandom-espidf", "slow_tests", "std", "test_logging", "unstable-testing-arm-no-hw", "unstable-testing-arm-no-neon", "wasm32_unknown_unknown_js"))' -C metadata=3807db18dbe2b831 -C extra-filename=-5890c4fcd540c9f4 --out-dir /home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps --target aarch64-unknown-linux-gnu -C linker=aarch64-linux-gnu-gcc -C strip=debuginfo -L dependency=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps -L dependency=/home/runner/work/ring/ring/target/release/deps --extern cfg_if=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/libcfg_if-311a393a154487db.rlib --extern getrandom=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/libgetrandom-4517178c3c9d757b.rlib --extern libc=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/liblibc-a35c490245773132.rlib --extern ring=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/libring-860b7f29bcca337c.rlib --extern untrusted=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/libuntrusted-a211e697d07dd481.rlib -L native=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/build/ring-9ec23736496dbc30/out`
   Compiling ring-cavp v0.1.0 (/home/runner/work/ring/ring/cavp)
     Running `CARGO=/home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_CRATE_NAME=shavs CARGO_MANIFEST_DIR=/home/runner/work/ring/ring/cavp CARGO_MANIFEST_PATH=/home/runner/work/ring/ring/cavp/Cargo.toml CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=ring-cavp CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 CARGO_SBOM_PATH='' CARGO_TARGET_TMPDIR=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/tmp LD_LIBRARY_PATH='/home/runner/work/ring/ring/target/release/deps:/home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' /home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name shavs --edition=2021 cavp/tests/shavs.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --emit=dep-info,link -C opt-level=3 -C lto -C codegen-units=1 -C overflow-checks=on --test --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=737e22b5137d5208 -C extra-filename=-c397c990b1305df2 --out-dir /home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps --target aarch64-unknown-linux-gnu -C linker=aarch64-linux-gnu-gcc -C strip=debuginfo -L dependency=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps -L dependency=/home/runner/work/ring/ring/target/release/deps --extern ring=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/libring-860b7f29bcca337c.rlib -L native=/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/build/ring-9ec23736496dbc30/out`
error: linking with `aarch64-linux-gnu-gcc` failed: exit status: 1
  |
  = note:  "aarch64-linux-gnu-gcc" "/tmp/rustcVcmlFx/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustcVcmlFx/{libring-860b7f29bcca337c.rlib}.rlib" "<sysroot>/lib/rustlib/aarch64-unknown-linux-gnu/lib/{libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lc" "-lm" "-lrt" "-lpthread" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcVcmlFx/raw-dylibs" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/build/ring-9ec23736496dbc30/out" "-L" "<sysroot>/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "/home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/hmac_tests-887166f8942c37c3" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-debug" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: /home/runner/work/ring/ring/target/aarch64-unknown-linux-gnu/release/deps/hmac_tests-887166f8942c37c3.hmac_tests.bb3648ddf19ab144-cgu.0.rcgu.o: unrecognized relocation type 0x13b in section `.rodata..Lswitch.table._ZN4core3ops8function6FnOnce9call_once17hcde9e8db08ba4659E.rel'
          /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: is this version of the linker - (GNU Binutils for Ubuntu) 2.38 - out of date ?
          /usr/lib/gcc-cross/aarch64-linux-gnu/11/../../../../aarch64-linux-gnu/bin/ld: final link failed: bad value
          collect2: error: ld returned 1 exit status

aarch64-linux-android

https://github.com/briansmith/ring/actions/runs/15145194221/job/42579103909

mk/cargo.sh +stable test -vv --target=aarch64-linux-android --no-run --release fails to link a test:

error: linking with `/usr/local/lib/android/sdk/ndk/27.2.12479018/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang` failed: exit status: 1
  |
  = note:  "/usr/local/lib/android/sdk/ndk/27.2.12479018/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang" "/tmp/rustc7iAwim/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustc7iAwim/{libring-28129c41485533d9.rlib}.rlib" "<sysroot>/lib/rustlib/aarch64-linux-android/lib/{libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lc" "-lm" "-ldl" "-llog" "-lunwind" "-ldl" "-lm" "-lc" "-L" "/tmp/rustc7iAwim/raw-dylibs" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/runner/work/ring/ring/target/aarch64-linux-android/release/build/ring-7346a50cac8b9043/out" "-o" "/home/runner/work/ring/ring/target/aarch64-linux-android/release/deps/pbkdf2_tests-3d083319a4f167f5" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-debug" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: ld.lld: error: /home/runner/work/ring/ring/target/aarch64-linux-android/release/deps/pbkdf2_tests-3d083319a4f167f5.pbkdf2_tests.81e0beb619f4e95d-cgu.0.rcgu.o:(.rodata..Lswitch.table._ZN4core3ops8function6FnOnce9call_once17h13d6622907b21806E.rel+0x0): unknown relocation (315) against symbol ring::digest::SHA1_FOR_LEGACY_USE_ONLY::h6a3dc428baeccf62
          ld.lld: error: /home/runner/work/ring/ring/target/aarch64-linux-android/release/deps/pbkdf2_tests-3d083319a4f167f5.pbkdf2_tests.81e0beb619f4e95d-cgu.0.rcgu.o:(.rodata..Lswitch.table._ZN4core3ops8function6FnOnce9call_once17h13d6622907b21806E.rel+0x4): unknown relocation (315) against symbol ring::digest::SHA256::he15e444f1113e84c
          ld.lld: error: /home/runner/work/ring/ring/target/aarch64-linux-android/release/deps/pbkdf2_tests-3d083319a4f167f5.pbkdf2_tests.81e0beb619f4e95d-cgu.0.rcgu.o:(.rodata..Lswitch.table._ZN4core3ops8function6FnOnce9call_once17h13d6622907b21806E.rel+0x8): unknown relocation (315) against symbol ring::digest::SHA384::he27a7edf8c3e9dc5
          ld.lld: error: /home/runner/work/ring/ring/target/aarch64-linux-android/release/deps/pbkdf2_tests-3d083319a4f167f5.pbkdf2_tests.81e0beb619f4e95d-cgu.0.rcgu.o:(.rodata..Lswitch.table._ZN4core3ops8function6FnOnce9call_once17h13d6622907b21806E.rel+0xc): unknown relocation (315) against symbol ring::digest::SHA512::h61f3561d70b10715
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

x86_64-apple-darwin

https://github.com/briansmith/ring/actions/runs/15143648178/job/42573984175

running 2 tests
test hmac_debug ... ok
test hmac_tests ... FAILED

failures:

---- hmac_tests stdout ----

thread 'hmac_tests' panicked at tests/hmac_tests.rs:83:9:
assertion `left == right` failed
  left: true
 right: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

(The test is checking whether the output of the HMAC computation is correct.)

[briansmith]

See tikv/tikv#18465 for another project that seems to have encountered the same error with the same unrecognized relocation type error on ARM64 Linux, using nightly-2025-04-04.

[briansmith]

The problem seems to be caused by LTO, as when I remove "lto = true" from the release profile in Cargo.toml, things are working. See https://github.com/briansmith/ring/actions/runs/15145798156.

[lqd]

Can you bisect the PR causing this with cargo-bisect-rustc? Thanks!

[briansmith]

Can you bisect the PR causing this with cargo-bisect-rustc? Thanks!

If you are asking me, then no. It is very unlikely I will get to it, though in normal circumstances I would try.

[lqd]

The bisection should be helpful regardless of who's doing it, it doesn't have to be you, no worries.