"llvm-ar: No such file or directory" in run-make-fulldeps tests with download-ci-llvm

[SimonSapin]

I recently added profile = "compiler" to config.toml in my local repository. In particular, this enables:

[llvm]
download-ci-llvm = "if-available"

When running the full test suite with ./x.py test I get three errors similar to this:

---- [run-make] run-make-fulldeps/cross-lang-lto stdout ----

error: make failed
status: exit code: 2
command: "make"
stdout:
------------------------------------------
LD_LIBRARY_PATH="/home/simon/projects/rust/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/cross-lang-lto/cross-lang-lto:/home/simon/projects/rust/build/x86_64-unknown-linux-gnu/stage1/lib:/home/simon/projects/rust/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/home/simon/projects/rust/build/x86_64-unknown-linux-gnu/stage0/lib" '/home/simon/projects/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc' --out-dir /home/simon/projects/rust/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/cross-lang-lto/cross-lang-lto -L /home/simon/projects/rust/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/cross-lang-lto/cross-lang-lto  lib.rs -Copt-level=2 -Clinker-plugin-lto -Ccodegen-units=1 --crate-type=staticlib -o /home/simon/projects/rust/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/cross-lang-lto/cross-lang-lto/liblib.a
(cd /home/simon/projects/rust/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/cross-lang-lto/cross-lang-lto; rm -f ./*.o; "/home/simon/projects/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin"/llvm-ar x  liblib.a)

------------------------------------------
stderr:
------------------------------------------
warning: ignoring --out-dir flag due to -o flag

warning: 1 warning emitted

/bin/sh: /home/simon/projects/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/llvm-ar: No such file or directory
make: *** [Makefile:23: staticlib] Error 127

Maybe the packaging of LLVM downloaded from CI needs to be fixed to include llvm-ar?

[camelid]

Cc @Mark-Simulacrum since you're the one who implemented this :)

[Mark-Simulacrum]

Yes, that seems right. I did not know we relied on the LLVM tooling -- probably there's a few more tools we need too, will need to check. I imagine it's mostly a matter of just adding them to this section: https://github.com/rust-lang/rust/blob/master/src/bootstrap/dist.rs#L2541-L2543, they should already be built.

I would be happy to take a PR for this.

[SimonSapin]

@Mark-Simulacrum I can make a patch adding a couple lines for llvm-ar like the ones there for llvm-config, but can I test it without landing it and waiting for the next Nightly?

(I assume testing locally involves compiling LLVM, so I might wait to do that until I have access again to a many-cores desktop rather than just my laptop.)

[Mark-Simulacrum]

Hm, sort of. So you will need to post a PR, but then bors try will generate sufficient artifacts to test. You will want to replace https://github.com/rust-lang/rust/blob/master/src/bootstrap/bootstrap.py#L435-L439 with the sha of the try build that bors posts.

[mati865]

llvm-nm might also be necessary but I'm not sure.

[SimonSapin]

@mati865 I’ll find out! I’ve already started a try build with just ar: #78131

@Mark-Simulacrum I can temporarily hard-code llvm_sha in src/bootstrap/bootstrap.py for the purpose of testing, but for everyone else the logic is still picking the last commit that touched the src/llvm-project submodule, which after #78131 lands will still be some commit that does not include #78131.

Would it make sense to have a "timestamp-ish" file, and pick the last commit that touched either the submodule or that file? That would only be useful until the next LLVM update, though. (Unless we need to tweak thepackaging of the rust-dev component again.)