Rustbuild: changing and testing libcore results in "found possibly newer version of crate `std` which `rand` depends on"

[LukasKalbertodt]

I tried testing libcore with the changes of #50466, which enables way faster testing of libcore and libstd.

When I execute ./x.py test --stage 0 --no-doc src/libcore the first time, everything works fine. If I then change test-code in libcore/tests, only the tests are rebuild and everything works (awesome!). But if I change libcore directly, it tries to rebuild a few things and then has a problem with the crate rand:

$ ./x.py test --stage 0 --no-doc src/libcore --test-args time
Updating only changed submodules
Submodules updated in 0.02 seconds
    Finished dev [unoptimized] target(s) in 0.0 secs
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling core v0.0.0 (file:///home/lukas/dev/rust/src/libcore)
   Compiling compiler_builtins v0.0.0 (file:///home/lukas/dev/rust/src/rustc/compiler_builtins_shim)
   Compiling libc v0.0.0 (file:///home/lukas/dev/rust/src/rustc/libc_shim)
   Compiling alloc v0.0.0 (file:///home/lukas/dev/rust/src/liballoc)
   Compiling std_unicode v0.0.0 (file:///home/lukas/dev/rust/src/libstd_unicode)
   Compiling unwind v0.0.0 (file:///home/lukas/dev/rust/src/libunwind)
   Compiling alloc_system v0.0.0 (file:///home/lukas/dev/rust/src/liballoc_system)
   Compiling panic_abort v0.0.0 (file:///home/lukas/dev/rust/src/libpanic_abort)
   Compiling alloc_jemalloc v0.0.0 (file:///home/lukas/dev/rust/src/liballoc_jemalloc)
   Compiling rustc_lsan v0.0.0 (file:///home/lukas/dev/rust/src/librustc_lsan)
   Compiling panic_unwind v0.0.0 (file:///home/lukas/dev/rust/src/libpanic_unwind)
   Compiling rustc_asan v0.0.0 (file:///home/lukas/dev/rust/src/librustc_asan)
   Compiling rustc_msan v0.0.0 (file:///home/lukas/dev/rust/src/librustc_msan)
   Compiling rustc_tsan v0.0.0 (file:///home/lukas/dev/rust/src/librustc_tsan)
   Compiling std v0.0.0 (file:///home/lukas/dev/rust/src/libstd)
    Finished release [optimized] target(s) in 30.15 secs
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling term v0.0.0 (file:///home/lukas/dev/rust/src/libterm)
   Compiling getopts v0.2.17
   Compiling test v0.0.0 (file:///home/lukas/dev/rust/src/libtest)
    Finished release [optimized] target(s) in 6.58 secs
Copying stage0 test from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Testing core stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling core v0.0.0 (file:///home/lukas/dev/rust/src/libcore)
error[E0460]: found possibly newer version of crate `std` which `rand` depends on
  --> libcore/../libcore/tests/lib.rs:53:1
   |
53 | extern crate rand;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: perhaps that crate needs to be recompiled?
   = note: the following crate versions were found:
           crate `std`: /home/lukas/dev/rust/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/libstd-5f3a747b9ee6cbda.rlib
           crate `std`: /home/lukas/dev/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-5f3a747b9ee6cbda.rlib
           crate `std`: /home/lukas/dev/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-5f3a747b9ee6cbda.so
           crate `std`: /home/lukas/dev/rust/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/libstd-5f3a747b9ee6cbda.so
           crate `rand`: /home/lukas/dev/rust/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/librand-89787ed220814c67.rlib

error: aborting due to previous error

For more information about this error, try `rustc --explain E0460`.
error: Could not compile `core`.
warning: build failed, waiting for other jobs to finish...
error: build failed


command did not execute successfully: "/home/lukas/dev/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-gnu" "-j" "8" "--release" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/home/lukas/dev/rust/src/libstd/Cargo.toml" "--lib" "--bins" "--examples" "--tests" "--benches" "-p" "core" "--" "time"
expected success, got: exit code: 101


failed to run: /home/lukas/dev/rust/build/bootstrap/debug/bootstrap test --stage 0 --no-doc src/libcore --test-args time
Build completed unsuccessfully in 0:00:44

Note that I can simply run x.py clean && x.py test --stage 0 --no-doc src/libcore and everything works again (and it doesn't even takes that much longer). But I guess this rand problem shouldn't occur anyway? The problem always occurs when performing the steps as described above.

CC @kennytm

[RalfJung]

I keep running into this as well.

A work-around is to do

rm build/*/stage0-std/*/release/deps/lib{rand,libc}-* -v

[RalfJung]

Also this does not just happen in stage 0. I also get the same problem in stage 1.

[AaronKutch]

I ran into this exact error running the same commands, but the workaround is not working for me.

$ ./x.py test --stage 0 --no-doc src/libcore
Updating only changed submodules
Submodules updated in 0.26 seconds
    Finished dev [unoptimized] target(s) in 0.30s
Building stage0 std artifacts (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu)
   Compiling libc v0.2.54
   Compiling backtrace-sys v0.1.27
   Compiling unwind v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libunwind)
   Compiling panic_abort v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libpanic_abort)
   Compiling backtrace v0.3.29
   Compiling panic_unwind v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libpanic_unwind)
   Compiling std v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libstd)
    Finished release [optimized] target(s) in 17.49s
Copying stage0 std from stage0 (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu / x86_64-pc-windows-gnu)
Building stage0 test artifacts (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu)
   Compiling unicode-width v0.1.5
   Compiling term v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libterm)
   Compiling proc_macro v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libproc_macro)
   Compiling getopts v0.2.19
   Compiling test v0.0.0 (C:\Users\Allen\Documents\GitHub\rust\src\libtest)
    Finished release [optimized] target(s) in 20.71s
Copying stage0 test from stage0 (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu / x86_64-pc-windows-gnu)
Testing core stage0 (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu)
   Compiling rand v0.6.1
error[E0460]: found possibly newer version of crate `std` which `rand_core` depends on
  --> C:\Users\Allen\.cargo\registry\src\github.com-1ecc6299db9ec823\rand-0.6.1\src\lib.rs:75:1
   |
75 | extern crate rand_core;
   | ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: perhaps that crate needs to be recompiled?
   = note: the following crate versions were found:
           crate `std`: \\?\C:\Users\Allen\Documents\GitHub\rust\build\x86_64-pc-windows-gnu\stage0-sysroot\lib\rustlib\x86_64-pc-windows-gnu\lib\libstd-62f505b9307d737b.rlib
           crate `std`: \\?\C:\Users\Allen\Documents\GitHub\rust\build\x86_64-pc-windows-gnu\stage0-std\x86_64-pc-windows-gnu\release\deps\libstd-62f505b9307d737b.rlib
           crate `std`: \\?\C:\Users\Allen\Documents\GitHub\rust\build\x86_64-pc-windows-gnu\stage0-sysroot\lib\rustlib\x86_64-pc-windows-gnu\lib\std-62f505b9307d737b.dll
           crate `std`: \\?\C:\Users\Allen\Documents\GitHub\rust\build\x86_64-pc-windows-gnu\stage0-std\x86_64-pc-windows-gnu\release\deps\std-62f505b9307d737b.dll
           crate `rand_core`: \\?\C:\Users\Allen\Documents\GitHub\rust\build\x86_64-pc-windows-gnu\stage0-std\x86_64-pc-windows-gnu\release\deps\librand_core-6c85245e34681cba.rlib

[scottmcm]

Nominating to try to bring this up to some team that could do something about this, since this is pretty annoying to everyone changing core/alloc/std and has been for over a year.

(I don't know what team, though, so apologies to infra if that's the wrong guess.)

[RalfJung]

@AaronKutch the work-around should delete \\?\C:\Users\Allen\Documents\GitHub\rust\build\x86_64-pc-windows-gnu\stage0-std\x86_64-pc-windows-gnu\release\deps\librand_core-6c85245e34681cba.rlib; if it doesn't that's likely a Windows thing and you'll have to figure out some other way to delete that file.

[AaronKutch]

It then complained about error[E0460]: found possibly newer version of crate core which winapi depends on. I tried deleting the libcore file, but that does not fix it.
Just to make sure, I don't have to run some cleaning command whenever I update my local clone of Rust? I would expect x.py to overwrite if necessary just like cargo does when updating the compiler.

[RalfJung]

It then complained about error[E0460]: found possibly newer version of crate core which winapi depends on. I tried deleting the libcore file, but that does not fix it.

Then go on deleting the winapi file that it mentions in the error. You have to get all of them cleared out; on Linux rand* is enough but the dependencies are different on Windows.

Just to make sure, I don't have to run some cleaning command whenever I update my local clone of Rust? I would expect x.py to overwrite if necessary just like cargo does when updating the compiler.

That's how it should go, but there's a bug. ;) You can do ./x.py clean to reset it entirely though.