Why does `x.py test src/libcore --stage 0` build everything up to the stage2 compiler?

[LukasKalbertodt]

Today I worked on libcore. To naive me, libcore should be just like any other library on crates.io. This means I would expect to switch into the directory and execute cargo test. Now I know that libcore is not exactly like any other library, but I still expect a few things. Things like "I don't need to build the stage2 compiler to run unit tests".

I tried to run this with the following result (I only kept the main steps):

$ ./x.py test src/libcore --stage 0
Updating only changed submodules
Submodules updated in 0.02 seconds
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage0 test from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage0 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage0 rustc from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building rustdoc for stage0 (x86_64-unknown-linux-gnu)
Building LLVM for x86_64-unknown-linux-gnu
Building stage0 codegen artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu, llvm)
Assembling stage1 compiler (x86_64-unknown-linux-gnu)
Building stage1 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage1 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage1 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage1 test from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage1 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage1 codegen artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu, llvm)
Assembling stage2 compiler (x86_64-unknown-linux-gnu)
Testing core stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)

It takes forever and doesn't make a lot of sense to me. I'd say I understand the basics of self hosting compilers, but I fail to understand x.py and I haven't found any resources that explain this behavior. Until now, I only worked on core or std and I always need to wait through long build times which also builds the compiler (multiple times). Also: if I change a thing in libcore, all of those steps are executed again! I don't understand...

I already experimented with the --stage and --keep-stage parameters, but apparently they are just broken somehow. Maybe we need a build system working group for all these issues? 😋

I know that you all are already trying to improve the compiler documentation a lot (thanks!), but I'm afraid that I think it's still very suboptimal when it comes to the build system (for reference, I read this and this). These long build times and in particular the feeling that everything could be faster "if stupid me would only understand the build system", are not really beneficial to attract (and keep) new contributors. There is only so much laundry one can during builds...

[ehuss]

I'm going to close this as I believe it is fixed. ./x.py test src/libcore --stage 0 should only use stage0 rustc/rustdoc now. It's still not perfect (there's currently a single doctest that fails since it requires a stage1 compiler), but should be good enough. Feel free to reopen if I misinterpreted!