Multiple codegen units breaks linking: "undefined reference to rust_begin_unwind"

[japaric]

STR

$ git clone https://github.com/clebi/stm32_magnetic

$ cd stm32_magnetic

$ git checkout 086b7d9deb1ae7bd3d82d63f65eb9430fab677c1

$ # disable incremental compilation to avoid other issues
$ echo '[profile.dev]' >> Cargo.toml
$ echo 'incremental = false' >> Cargo.toml

$ # discard the Xargo sysroot to avoid problems with stale artifacts
$ rm -rf ~/.xargo

$ # this uses multiple codegen units
$ xargo rustc -- -Z print-link-args > link-args
(..)
  = note: /home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib/libcore-82bb515fb6a7673a.rlib(core-82bb515fb6a7673a.core13-a2b2f175d842448d578bfd1df50c2091.rs.rcgu.o): In function `core::panicking::panic_fmt':
          /home/japaric/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/panicking.rs:71: undefined reference to `rust_begin_unwind'

$ cat link-args | tr ' ' '\n'
"arm-none-eabi-ld"
"-L"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic0.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic1.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic10.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic11.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic12.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic13.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic14.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic15.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic2.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic3.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic4.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic5.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic6.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic7.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic8.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic9.rcgu.o"
"-o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d"
"--gc-sections"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/debug/deps"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/build/cortex-m-rt-67ae64ff3b026a1b/out"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/build/f3-9427da8830940471/out"
"-L"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib"
"-Bstatic"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m_rtfm-c0937f58fbde0459.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/librtfm_core-1e85456d9189c8f2.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libuntagged_option-cb0ee6b31e8b5540.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libf3-5f30af66a432ec74.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libstm32f30x-5ab99d65170ba613.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m_rt-a656f43a1b217a17.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libr0-eea37c5929500808.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libstatic_ref-4e80cdb2ce553b2f.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libembedded_hal-a5c5f16836b1d4fa.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcast-c3830481e6a0787e.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libnb-2c1e61e97adbfc28.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m-8e97aa6b5ac0c1c3.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libbare_metal-c2470b1735ede45e.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libaligned-ca7b9812ff225cca.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libvolatile_register-a39da4476292d8c3.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libvcell-72540773165f79da.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m_semihosting-d2a306addcb7fb3a.rlib"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib/libcore-82bb515fb6a7673a.rlib"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib/libcompiler_builtins-21afa2c8d0d459e7.rlib"
"-Tlink.x"
"-Bdynamic"

$ # the linker command succeeds if the objects / archives are wrapped in --start-group / --end-group
$ "arm-none-eabi-ld" --start-group "-L" (..) "-Bdynamic" --end-group && echo OK
OK

$ # the build also succeeds if a single codegen unit is used
$ echo 'codegen-units = 1' >> Cargo.toml
$ rm -rf ~/.xargo && cargo clean
$ xargo build && echo OK
OK

Meta

$ rustc -V
rustc 1.24.0-nightly (77e189cd7 2017-12-28)

cc @alexcrichton @clebi

[japaric]

After bisecting it seems that this broke between nightly-2017-12-27 (good) and nightly-2017-12-28 (bad).

[alexcrichton]

@japaric can you confirm where, for the embedded platform here, the symbol rust_begin_unwind is defined? I think this may be the same as #18807 (an ancient bug now!) where rustc doesn't understand this special symbol relationship well enough to pass the right linker flags.

[japaric]

@alexcrichton the panic_fmt lang item, and several other lang items, is defined in the cortex-m-rt crate.

[alexcrichton]

Ok cool, makes sense! In that case I believe this is indeed the same issue as #18807.

The compiler knows who needs rust_begin_unwind (libcore) and also knows who defines it (libstd in general and cortex-m-rt in your case). It's the compiler's job to understand that this is a cyclic dependency and insert --start-group and --end-group appropriately but it currently does not.

I believe the correct fix here (and for #18807) is for rustc to detect which crate defines rust_begin_unwind and which crate requires it (std/core in general) and to automatically insert --start-group and --end-group around these libraries. That should be enough to make the linker error go away.

[japaric]

ugh, I'm now seeing this linker error in another project when compiling it using the dev profile even though codegen-units is set to 1 and incremental is disabled in the Cargo.toml. My current fix is ... to enable LTO on the dev profile.

@alexcrichton any chance you'll have to time soon to implement the fix you mentioned? Otherwise I can look into it next week, probably.

[alexcrichton]

@japaric unfortunately I don't think I'll have time to look into this, but I can certainly help out with questions if they come up!

[japaric]

@alexcrichton alright. I'll ping you on IRC when I get a chance to look into this.