Cannot bootstrap rust with system compiler and customized libdir in 1.89 (stage0-sysroot)

[lucabonnix]

Summary

In 1.89 the patch from #141729, related to #141722, was applied but the build got worst since it stops just after building boostrap if a custom libdir (e.g. "lib64") was used.

Command used

python3 x.py build

Expected behaviour

The build system should create libraries under stage0-sysroot/lib64/rustlib and not try to make any link to stage0-sysroot/lib/rustlib

Actual behaviour

The boostrap stage failed to make link from ../lib64/rustlib to stage0-sysroot/lib/rustlib because it is an existing directory.

Bootstrap configuration (bootstrap.toml)

# Avoid triggering "git --local" on non-git vendored sources:
profile = "dist"

[llvm]
link-shared = true
download-ci-llvm = false

[build]
build = "x86_64-unknown-linux-gnu"
host = ["x86_64-unknown-linux-gnu"]
target = ["x86_64-unknown-linux-gnu"]
tools = ["analysis", "cargo", "clippy", "rustfmt", "src", "rust-analyzer", "rust-demangler"]
submodules = false
vendor = true
extended = true
rustc = "/usr/bin/rustc"
cargo = "/usr/bin/cargo"
profiler = true
sanitizers = true
# Do not query new versions of dependencies online.
locked-deps = true
docs = false
jobs =  8

[install]
prefix = "/usr"
docdir = "doc/rust-1.89.0"
libdir = "lib64"
mandir = "man"

[rust]
codegen-units = 0
channel = "stable"
rpath = false
codegen-tests = false

# Add this stuff to build with the system LLVM:
[target.i586-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.i686-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.x86_64-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.arm-unknown-linux-gnueabi]
llvm-config = "/usr/bin/llvm-config"

[target.armv5te-unknown-linux-gnueabi]
llvm-config = "/usr/bin/llvm-config"

[target.armv7-unknown-linux-gnueabihf]
llvm-config = "/usr/bin/llvm-config"

[target.aarch64-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.powerpc-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.powerpc64-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.powerpc64le-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.mips-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.mipsel-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.mips64el-unknown-linux-gnuabi64]
llvm-config = "/usr/bin/llvm-config"

[target.riscv64gc-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.s390x-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

[target.loongarch64-unknown-linux-gnu]
llvm-config = "/usr/bin/llvm-config"

Operating system

Slackware current (developing branch 15.1)

HEAD

2948388

Additional context

The failed link was /root/tmp/rustc-1.89.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib => ../lib64/rustlib

Build Log

Building bootstrap
   Compiling proc-macro2 v1.0.89
   Compiling memchr v2.7.4
   Compiling unicode-ident v1.0.13
   Compiling typenum v1.17.0
   Compiling version_check v0.9.5
   Compiling libc v0.2.172
   Compiling shlex v1.3.0
   Compiling crossbeam-utils v0.8.20
   Compiling regex-syntax v0.8.5
   Compiling rustix v1.0.7
   Compiling serde v1.0.215
   Compiling pkg-config v0.3.31
   Compiling linux-raw-sys v0.9.3
   Compiling cfg-if v1.0.0
   Compiling bitflags v2.6.0
   Compiling anstyle v1.0.10
   Compiling heck v0.5.0
   Compiling clap_lex v0.7.2
   Compiling serde_json v1.0.132
   Compiling cc v1.2.23
   Compiling generic-array v0.14.7
   Compiling clap_builder v4.5.20
   Compiling same-file v1.0.6
   Compiling semver v1.0.23
   Compiling log v0.4.22
   Compiling walkdir v2.5.0
   Compiling itoa v1.0.11
   Compiling cpufeatures v0.2.15
   Compiling ryu v1.0.18
   Compiling bootstrap v0.0.0 (/root/tmp/rustc-1.89.0-src/src/bootstrap)
   Compiling termcolor v1.4.1
   Compiling home v0.5.9
   Compiling crossbeam-epoch v0.9.18
   Compiling aho-corasick v1.1.3
   Compiling object v0.36.5
   Compiling quote v1.0.37
   Compiling crossbeam-deque v0.8.5
   Compiling syn v2.0.87
   Compiling filetime v0.2.25
   Compiling cmake v0.1.54
   Compiling lzma-sys v0.1.20
   Compiling block-buffer v0.10.4
   Compiling crypto-common v0.1.6
   Compiling xz2 v0.1.7
   Compiling digest v0.10.7
   Compiling regex-automata v0.4.9
   Compiling sha2 v0.10.8
   Compiling xattr v1.5.0
   Compiling fd-lock v4.0.4
   Compiling tar v0.4.43
   Compiling toml v0.5.11
   Compiling clap_derive v4.5.18
   Compiling serde_derive v1.0.215
   Compiling bstr v1.10.0
   Compiling globset v0.4.15
   Compiling opener v0.5.2
   Compiling ignore v0.4.23
   Compiling clap v4.5.20
   Compiling clap_complete v4.5.37
   Compiling build_helper v0.1.0 (/root/tmp/rustc-1.89.0-src/src/build_helper)
    Finished `dev` profile [unoptimized] target(s) in 56.03s
fatal: not a git repository (or any of the parent directories): .git
WARNING: The `change-id` is missing in the `bootstrap.toml`. This means that you will not be able to track the major changes made to the bootstrap configurations.
NOTE: to silence this warning, add `change-id = 142379` or `change-id = "ignore"` at the top of `bootstrap.toml`

thread 'main' panicked at src/bootstrap/src/lib.rs:1715:17:
self.symlink_file(link, dst) failed with File exists (os error 17)
stack backtrace:
   0:     0x55933de87b3b - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::he45f3ccaa52a99d9
   1:     0x55933deb4af3 - core::fmt::write::h94d58bb5cce5ed6d
   2:     0x55933de7d30f - std::io::Write::write_fmt::h77f3b0181356a8aa
   3:     0x55933de87983 - std::sys::backtrace::BacktraceLock::print::h08bfa47a7be47d51
   4:     0x55933de66746 - std::panicking::default_hook::{{closure}}::h78c1501885fc5a90
   5:     0x55933de665c9 - std::panicking::default_hook::h0692636975d00898
   6:     0x55933de66c6f - std::panicking::rust_panic_with_hook::h55abdcd417182549
   7:     0x55933de8877a - std::panicking::begin_panic_handler::{{closure}}::h14873d49c4bec08c
   8:     0x55933de87d69 - std::sys::backtrace::__rust_end_short_backtrace::hdd0b1fa3983ca37f
   9:     0x55933de667dd - __rustc[b723cc1e8dfe1f7e]::rust_begin_unwind
  10:     0x55933d147560 - core::panicking::panic_fmt::hb346e6e910242923
  11:     0x55933d7c05c2 - bootstrap::Build::copy_link_internal::h976ed895cfaeda1f
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/utils/helpers.rs:55:23
  12:     0x55933d7bfd08 - bootstrap::Build::copy_link::h04907fb2f624966e
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/lib.rs:1677:9
  13:     0x55933d7c1971 - bootstrap::Build::cp_link_r::h201e9dff046c63a3
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/lib.rs:1752:17
  14:     0x55933d872690 - <bootstrap::core::build_steps::compile::StdLink as bootstrap::core::builder::Step>::run::h22c67eaea5f1e003
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/build_steps/compile.rs:810:13
  15:     0x55933d2e467e - bootstrap::core::builder::Builder::ensure::h2111da69f53fcedb
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1596:23
  16:     0x55933d86b826 - <bootstrap::core::build_steps::compile::Std as bootstrap::core::builder::Step>::run::h140b2303c6fd5616
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/build_steps/compile.rs:156:13
  17:     0x55933d2dddfe - bootstrap::core::builder::Builder::ensure::h17c95c6165f7084f
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1596:23
  18:     0x55933d874e03 - <bootstrap::core::build_steps::compile::Rustc as bootstrap::core::builder::Step>::run::hfd0f3e25a13cb185
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/build_steps/compile.rs:1070:9
  19:     0x55933d2dcc42 - bootstrap::core::builder::Builder::ensure::h1765abc139048470
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1596:23
  20:     0x55933d87e656 - <bootstrap::core::build_steps::compile::Assemble as bootstrap::core::builder::Step>::run::hb9cdeb4199072439
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/build_steps/compile.rs:2151:28
  21:     0x55933d2ffaeb - bootstrap::core::builder::Builder::ensure::h34ff2e98ba5b4839
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1596:23
  22:     0x55933d854226 - bootstrap::core::builder::Builder::compiler::h69ec755314fe17d7
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1298:9
  23:     0x55933d87dbae - <bootstrap::core::build_steps::compile::Assemble as bootstrap::core::builder::Step>::run::hb9cdeb4199072439
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/build_steps/compile.rs:2119:13
  24:     0x55933d2ffaeb - bootstrap::core::builder::Builder::ensure::h34ff2e98ba5b4839
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1596:23
  25:     0x55933d854226 - bootstrap::core::builder::Builder::compiler::h69ec755314fe17d7
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1298:9
  26:     0x55933d86b523 - <bootstrap::core::build_steps::compile::Std as bootstrap::core::builder::Step>::make_run::h3f5b5902cc3b56d1
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/build_steps/compile.rs:123:23
  27:     0x55933d84c732 - bootstrap::core::builder::StepDescription::maybe_run::h7e95f838cd642301
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:475:13
  28:     0x55933d84d928 - bootstrap::core::builder::StepDescription::run::h331e17935a42450d
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:527:21
  29:     0x55933d854153 - bootstrap::core::builder::Builder::run_step_descriptions::h6e9d2923300ece2f
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1271:9
  30:     0x55933d853ea2 - bootstrap::core::builder::Builder::execute_cli::h17987e1d3b98e166
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/core/builder/mod.rs:1251:9
  31:     0x55933d7b95db - bootstrap::Build::build::h3760a0dbee2832e4
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/lib.rs:705:17
  32:     0x55933d150d2a - bootstrap::main::h8816bafb53051c27
                               at /root/tmp/rustc-1.89.0-src/src/bootstrap/src/bin/main.rs:99:5
  33:     0x55933d14f5d3 - core::ops::function::FnOnce::call_once::h0d766f98b6d74dae
                               at /tmp/rustc-1.88.0-src/library/core/src/ops/function.rs:250:5
  34:     0x55933d147f16 - std::sys::backtrace::__rust_begin_short_backtrace::h9f7b42410b3f6c79
                               at /tmp/rustc-1.88.0-src/library/std/src/sys/backtrace.rs:152:18
  35:     0x55933d14cc29 - std::rt::lang_start::{{closure}}::h9a27c06523cd650f
                               at /tmp/rustc-1.88.0-src/library/std/src/rt.rs:199:18
  36:     0x55933de7a6b5 - std::rt::lang_start_internal::h964efab8d10efc22
  37:     0x55933d14cc11 - std::rt::lang_start::h80d1d1e3693ffef3
                               at /tmp/rustc-1.88.0-src/library/std/src/rt.rs:198:5
  38:     0x55933d15297e - main
  39:     0x7f2bb702ad14 - __libc_start_call_main
  40:     0x7f2bb702adc6 - __libc_start_main@GLIBC_2.2.5
  41:     0x55933d147c31 - _start
                               at /tmp/glibc-2.42/csu/../sysdeps/x86_64/start.S:115:0
  42:                0x0 - <unknown>
Panic was initiated from src/bootstrap/src/lib.rs:1715:17
Build completed unsuccessfully in 0:00:57

[Kobzol]

We plan to get rid of stage0-sysroot to avoid problems like this, but it will likely take a few months.

[lucabonnix]

It seems that in rust 1.89 the "Building stage0 library artifacts (x86_64-unknown-linux-gnu)" is missing if using system compiler, so it cannot have core/std libraries to build stage1.

Is there any way to force "Building stage0 library artifacts"?

[Kobzol]

Building stage0 std is no longer a thing, we have changed the bootstrap sequence (https://blog.rust-lang.org/inside-rust/2025/05/29/redesigning-the-initial-bootstrap-sequence/).

[lucabonnix]

I get a deeper look to the build system, and it (useless) copies the system /usr/lib to local stage0-sysroot

If the system rustc is used, it would be only necessary to copy (or link) files under (e.g. for my system: /usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib):

rustc --print target-libdir

and they (or simply the parent dir) should be copied/linked to to the correct stage0-sysroot libdir (stage0-sysroot/lib64/rustlib/x86_64-unknown-linux-gnu/lib in my system).

Currently the build system, regardless the target-libdir, copies all files/directories under /usr/lib to stage0-sysroot/lib/, that is a quite useless copy of some GB.

The build error on my system was due to the link /usr/lib/rustlib => ../lib64/rustlib that I created in the past to try to fix the build, but after removing the link, building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu) fails because it did not find libraries under stage0-sysroot/lib64/rustlib/x86_64-unknown-linux-gnu/lib.

If I make a symbolic link stage0-sysroot/lib64/rustlib/x86_64-unknown-linux-gnu/lib => /usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib just after the build system has created the stage0-sysroot tree, all works fine!!

[Kobzol]

The problem is that we need not only the target libdir, but also the host libdir (because rustc links to these dynamic libraries using rpath with a relative path). And the host libdir in your case is actually /usr/lib. And since we don't know which exact libraries are required, we copy everything.

The whole concept of the stage0-sysroot directory is flawed, and I want to get rid of it; I'm actively working on it at the moment.