Skip to content

ThinLTO and -o compiler option can lead to duplicate object file inclusion in staticlib #64153

Closed

Description

I've verified this on macOS 10.14/Xcode 10.3 as well as Ubuntu 14.04/GCC 4.8
rustc -V: rustc 1.36.0-nightly (50a0def 2019-05-21) (although the code i mention is the same in master)

tl;dr object files (.o) prefixes are inferred from -o, but lto object file exclusion uses crate name. External build systems which use -o can face build failures.

Setup:

  1. A staticlib that depends on a rlib (standard crate)
  2. Build using lto=thin.
  3. The rlib is built with -o that causes it to have a custom output name.

Sample: https://gist.github.com/nikhilm/a72d89002553ecab4511fbe77df223cc
Run cargo build --release -vv to get the command lines.

First, just building the dependent rlib (by copying the cargo invocation, no -o passed)

$ ar t target/release/deps/libbitflags-9e9e338798b0f64c.rlib
__.SYMDEF
bitflags-9e9e338798b0f64c.bitflags.e90hrf8b-cgu.0.rcgu.o
rust.metadata.bin
bitflags-9e9e338798b0f64c.bitflags.e90hrf8b-cgu.0.rcgu.bc.z

notice how the prefix is bitflags

Now, tweaking the rustc command for bitflags to add a -o and running ar on that file:

__.SYMDEF
libbitflags-9e9e338798b0f64c.bitflags.e90hrf8b-cgu.0.rcgu.o
rust.metadata.bin
libbitflags-9e9e338798b0f64c.bitflags.e90hrf8b-cgu.0.rcgu.bc.z

notice how the prefix is libbitflags.

This is fine. But when compiling a staticlib comprising this rlib, there is a special case when lto is enabled.

if lto && fname.starts_with(&obj_start) && fname.ends_with(".o") {
, where the obj_start is the name of the crate (bitflags).

Now, if we take a look at the cargo generated staticlib, running ar -t target/release/libhello_world.a:

__.SYMDEF
hello_world-bdf2887e2bd4c34b.3v2guokzdntel6g4.rcgu.o
hello_world-bdf2887e2bd4c34b.alloc.a2n556im-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.backtrace_sys.cepblb1k-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.bitflags.e90hrf8b-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.core.ec2x3rcv-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.hashbrown.894rc464-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.hello_world.7socix02-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.hello_world.7socix02-cgu.1.rcgu.o
hello_world-bdf2887e2bd4c34b.libc.5av1bc7k-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.panic_unwind.354k3km2-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.rustc_demangle.cq0g9zqx-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.rustc_std_workspace_alloc.eehx1vq7-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.rustc_std_workspace_core.bloqy3w9-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.std.7g3y6w1q-cgu.0.rcgu.o
hello_world-bdf2887e2bd4c34b.unwind.5mw5tvii-cgu.0.rcgu.o
...

However, if we tweak the rustc for the staticlib to link against the rlib generated using -o
If this is different from the crate name, we are in trouble because the starts_with match fails (libbitflags does not start with bitflags).

__.SYMDEF
libhello_world-bdf2887e2bd4c34b.3v2guokzdntel6g4.rcgu.o
libhello_world-bdf2887e2bd4c34b.alloc.a2n556im-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.backtrace_sys.cepblb1k-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.bitflags.e90hrf8b-cgu.0.rcgu.o  <--------
libhello_world-bdf2887e2bd4c34b.core.ec2x3rcv-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.hashbrown.894rc464-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.hello_world.7socix02-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.hello_world.7socix02-cgu.1.rcgu.o
libhello_world-bdf2887e2bd4c34b.libc.5av1bc7k-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.panic_unwind.354k3km2-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.rustc_demangle.cq0g9zqx-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.rustc_std_workspace_alloc.eehx1vq7-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.rustc_std_workspace_core.bloqy3w9-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.std.7g3y6w1q-cgu.0.rcgu.o
libhello_world-bdf2887e2bd4c34b.unwind.5mw5tvii-cgu.0.rcgu.o
libbitflags-9e9e338798b0f64c.bitflags.e90hrf8b-cgu.0.rcgu.o    <------- symbols going to conflict with the above
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Labels

A-linkageArea: linking into static, shared libraries and binariesO-linuxOperating system: LinuxO-macosOperating system: macOSP-highHigh priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions