Rustc does not warn when .so/.rlib files are not in sync

[alexcrichton]

Right now the logic for reading metadata for a crate is to read it the first time the crate's dylib or rlib file is found. On the second file that's found, the metadata isn't read at all because metadata reading is one of the known slow portions of the compiler.

This implies that if an rlib file is regenerated, but the dylib isn't, then the two are out of sync and rustc will read the old metadata from the dylib and link to the rlib, resulting in linker errors.

I don't want to slow down rustc by reading metadata from all rlibs and dylibs, but this situation is far less than optimal. Perhaps this is a job for a rustpkg-like tool?

[emberian]

@alexcrichton is this still relevant? we don't do this checking, but the metadata isn't pulled from the dylib.

[alexcrichton]

This has an interesting development:

$ RUST_LOG=rustc::metadata::loader=info rustc foo.rs 2>&1 | grep '^reading'
reading libstd-aad93cea-0.11-pre.rlib => 0ms
reading libstd-aad93cea-0.11-pre.so => 47ms
reading liblibc-8f65d5a8-0.10-pre.rlib => 0ms
reading liblibc-8f65d5a8-0.10-pre.so => 0ms
reading libstd-aad93cea-0.11-pre.rlib => 0ms
reading libstd-aad93cea-0.11-pre.so => 46ms
reading liblibc-8f65d5a8-0.10-pre.rlib => 0ms
reading liblibc-8f65d5a8-0.10-pre.so => 0ms
reading libnative-83574243-0.11-pre.rlib => 0ms
reading libnative-83574243-0.11-pre.so => 1ms

This issue is supposed to be relevant, but it looks like we're reading metadata of rlibs and dylibs. This is not necessary, and it looks like we could shave ~100ms off compiling hello world if it were taken care of.

In short, I'd like to keep this issue open.

[reem]

Triage: I see that some progress has been made on this issue in the above commits. Does that re-enable this issue?

[steveklabnik]

$ RUST_LOG=rustc::metadata::loader=info rustc hello.rs 2>&1 | grep reading
INFO:rustc::metadata::loader: rlib reading metadata from: /home/steve/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-4e7c5e5c.rlib
INFO:rustc::metadata::loader: reading libstd-4e7c5e5c.rlib => 0ms

just rlibs now, apparently. I'm giving this a close, @alexcrichton , feel free to reopen.

[alexcrichton]

I think this is still an issue (reopening)

In this test case below a blank file is compiled to an rlib/dylib pair, then the rlib is recompiled with a new function. A later executable is then linked against the dylib (using the rlib metadata), succeeding all the way until the linker actually fails.

$ touch foo.rs
$ rustc --crate-type dylib,rlib foo.rs
$ ls
foo.rs       libfoo.dylib libfoo.rlib
$ echo 'pub fn foo() {}' > foo.rs
$ rustc --crate-type rlib foo.rs
$ echo 'extern crate foo; fn main() { foo::foo(); }' > bar.rs
$ rustc bar.rs -L. -C prefer-dynamic
error: linking with `cc` failed: exit code: 1
note: cc '-m64' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-o' 'bar' 'bar.o' '-Wl,-force_load,/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib/libmorestack.a' '-Wl,-dead_strip' '-nodefaultlibs' '-L' '/Users/acrichton/test' '-lfoo' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-lstd-4e7c5e5c' '-L' '.' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-L' '/Users/acrichton/test/.rust/lib/x86_64-apple-darwin' '-L' '/Users/acrichton/test/lib/x86_64-apple-darwin' '-L' '/Users/acrichton/.rust/lib/x86_64-apple-darwin' '-lSystem' '-lpthread' '-lc' '-lm' '-lcompiler-rt'
note: ld: warning: directory not found for option '-L/Users/acrichton/test/.rust/lib/x86_64-apple-darwin'
ld: warning: directory not found for option '-L/Users/acrichton/test/lib/x86_64-apple-darwin'
ld: warning: directory not found for option '-L/Users/acrichton/.rust/lib/x86_64-apple-darwin'
Undefined symbols for architecture x86_64:
  "foo::ha37403d109b4e734eaa", referenced from:
      main::h7e1981c1a43d0b7afaa in bar.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: aborting due to previous error

[steveklabnik]

Ahh, I forgot the crate-types. :(

[steveklabnik]

Triage: the error is slightly different:

error: linking with `cc` failed: exit code: 1
note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/steve/.multirust/toolchains/nightly/lib/rustlib/x86_64-unknown-linux-gnu/lib" "bar.0.o" "-o" "bar" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "." "-L" "/home/steve/.multirust/toolchains/nightly/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,-Bdynamic" "-L" "/home/steve/tmp" "-l" "foo" "-L" "/home/steve/.multirust/toolchains/nightly/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "std-fd663c41" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "rt" "-l" "compiler-rt"
note: bar.0.o: In function `main::h7afac322a1b32d81faa':
bar.0.rs:(.text._ZN4main20h7afac322a1b32d81faaE+0x2): undefined reference to `foo::hfe26232f3a6cc9a1eaa'
collect2: error: ld returned 1 exit status

error: aborting due to previous error