Serde with serde_derive fails to build in "alt" compiler on OS X

[SimonSapin]

Steps to reproduce:

cargo new a
cd a
echo 'serde = {version = "1.0", features = ["serde_derive"]}' >> Cargo.toml
cargo build

Alternative steps to reproduce that will provides a suitable compiler:

# This branch is temporary,
# it will disappear once this issue is resolved and later be re-purposed
git clone https://github.com/servo/servo/ --branch rustup
cd servo
./mach cargo build -p serde

On OS X, with LLVM assertions disabled (like in rustc builds added in #39754, which are the default in Servo’s mach), rustc 1.20.0-nightly (69c65d2 2017-06-28) fails with:

error: dlopen(/private/tmp/a/target/debug/deps/libserde_derive-73114808ba8f6d5d.dylib, 1): Library not loaded: @rpath/libstd-82587804d4836e27.dylib
  Referenced from: /private/tmp/a/target/debug/deps/libserde_derive-73114808ba8f6d5d.dylib
  Reason: image not found
   --> /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs:261:1
    |
261 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `serde`.

This problem does not occur with nightly-2017-06-29 from rustup, which has LLVM assertions enabled. (But perhaps that’s not the only difference between the two kinds of builds?)

[SimonSapin]

perhaps that’s not the only difference between the two kinds of builds?

What’s RUST_CONFIGURE_ARGS="--enable-extended"?
https://github.com/rust-lang/rust/pull/39754/files#diff-354f30a63fb0907d4ad57269548329e3R75

[SimonSapin]

It’s also enabled in the "non-alt" build, so probably not relevant.

[kennytm]

Have you tried building the project in the same disk partition where you've installed rustc, i.e. inside /Users/...?

[SimonSapin]

Yes. With the second set of steps (and -v added), the error message looks like this:

   Compiling serde v1.0.8
     Running `rustc --crate-name serde /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs --crate-type lib --emit=dep-info,metadata -C codegen-units=4 -C debuginfo=2 --cfg 'feature="std"' --cfg 'feature="rc"' --cfg 'feature="derive"' --cfg 'feature="default"' --cfg 'feature="serde_derive"' -C metadata=b12802d89eb13761 -C extra-filename=-b12802d89eb13761 --out-dir /Users/simon/projects/servo/target/debug/deps -L dependency=/Users/simon/projects/servo/target/debug/deps --extern serde_derive=/Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib --cap-lints allow -W unused-extern-crates`
error: dlopen(/Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib, 1): Library not loaded: @rpath/libstd-82587804d4836e27.dylib
  Referenced from: /Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib
  Reason: image not found
   --> /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs:261:1
    |
261 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `serde`.

[kennytm]

Does DYLD_PRINT_RPATHS=1 cargo build show anything special?

[SimonSapin]

@kennytm not as far I can tell.

[kennytm]

Okay... There are many other debugging options for dyld but I doubt if they are useful.

The problem is that @rpath/libstd-82587804d4836e27.dylib is not found, meaning @rpath/ is not expanded properly. I've checked libserde_derive-******.dylib does not have an LC_RPATH load command, meaning the @rpath is only supplied through the environment variable DYLD_LIBRARY_PATH set by Cargo.

I don't think the alt-build's and normal build's Cargo has any difference though.

Verify if rustc --print=sysroot is correct?

[SimonSapin]

./mach rustc --print=sysroot with and without llvm-assertions = true in .servobuild:

/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc
/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin/rustc

Both contain:

bin		lib		manifest.in	share

[kennytm]

Both contains libstd-82587804d4836e27.dylib directly inside lib/? (not lib/rustlib/<target>/lib/)

[SimonSapin]

Almost, the -alt one has a different hash:

$ ls /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c*/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd*
/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-5e0ec722ea8df7e6.dylib
/Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-82587804d4836e27.dylib

[kennytm]

Huh, that's even stranger. The @rpath should be resolved correctly. But the alt- rustc linked serde_derive using the libstd from the non-alt-build??

[SimonSapin]

Well it sounds unlikely to me that “from the other build” is really what’s going on, I’d bet on incorrect hashing. How is the hexadecimal part of libstd-82587804d4836e27.dylib determined? What base values are being hashed?

[kennytm]

IIRC rustc will just search for all libstd-*, it doesn't recompute the hash. Would it work if you move /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/ out of /Users/simon/tmp, and then do a cargo clean + build?

[SimonSapin]

I first hit the issue before the "non-alt" build was downloaded.