Build from source with autodiff fails on Mac OS

[kevinyamauchi]

Summary

I am trying to build rustc with autodiff support using the instructions in the guide. The build is currently failing on my M2 Macbook. I would appreciate any guidance on how to get the build to complete. Thanks!

Command used

git clone git@github.com:rust-lang/rust
cd rust
./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-assertions --enable-option-checking --disable-docs --set llvm.download-ci-llvm=true

RUST_BACKTRACE=1 ./x build -v --stage 1 library | tee build_output.txt

I noticed that the suggested configuration was updated recently (#147888), so I also tried the configuration below and obtained the same result.

./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs

Expected behaviour

The build completes without errors.

Actual behaviour

The build fails with a RuntimeError.

    Finished `release` profile [optimized] target(s) in 3.45s
using sysroot /Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1
Dirty - /Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1-std
running: DYLD_LIBRARY_PATH="/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/lib" "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" "--target" "aarch64-apple-darwin" "--print=deployment-target" (failure_mode=Exit) (created at src/bootstrap/src/core/builder/mod.rs:1614:23, executed at src/bootstrap/src/core/build_steps/compile.rs:538:26)
rustc -Cllvm-args="..." with: Unknown command line argument '-mergefunc-use-aliases'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--riscv-no-aliases'?
rustc -Cllvm-args="..." with: Unknown command line argument '-preserve-alignment-assumptions-during-inlining=false'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--mips-align-constant-islands=false'?
rustc -Cllvm-args="..." with: Unknown command line argument '-import-cold-multiplier=0.1'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--ignore-xcoff-visibility=0.1'?
Command `DYLD_LIBRARY_PATH="/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/lib" "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" "--target" "aarch64-apple-darwin" "--print=deployment-target" (failure_mode=Exit)` failed with exit code 1
Created at: src/bootstrap/src/core/builder/mod.rs:1614:23
Executed at: src/bootstrap/src/core/build_steps/compile.rs:538:26

--- STDOUT vvv


Traceback (most recent call last):
  File "/Users/kyamauch/Documents/rust_source/rust/x.py", line 53, in <module>
    bootstrap.main()
  File "/Users/kyamauch/Documents/rust_source/rust/src/bootstrap/bootstrap.py", line 1410, in main
    bootstrap(args)
  File "/Users/kyamauch/Documents/rust_source/rust/src/bootstrap/bootstrap.py", line 1370, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/Users/kyamauch/Documents/rust_source/rust/src/bootstrap/bootstrap.py", line 237, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /Users/kyamauch/Documents/rust_source/rust/build/bootstrap/debug/bootstrap build -v --stage 1 library

Bootstrap configuration (bootstrap.toml)

# Use different pre-set defaults than the global defaults.
#
# See `src/bootstrap/defaults` for more information.
# Note that this has no default value (x.py uses the defaults in `bootstrap.example.toml`).
profile = 'dist'

[llvm]

# Currently, we only support this when building LLVM for the build triple.
#
# Note that many of the LLVM options are not currently supported for
# downloading. Currently only the "assertions" option can be toggled.
download-ci-llvm = true

# Indicates whether the LLVM assertions are enabled or not
# NOTE: When assertions are disabled, bugs in the integration between rustc and LLVM can lead to
# unsoundness (segfaults, etc.) in the rustc process itself, not just in the generated code.
assertions = true

# Whether to build Enzyme as AutoDiff backend.
enzyme = true

[gcc]

[build]

# Whether to build documentation by default. If false, rustdoc and
# friends will still be compiled but they will not be used to generate any
# documentation.
#
# You can still build documentation when this is disabled by explicitly passing paths,
# e.g. `x doc library`.
docs = false

# Arguments passed to the `./configure` script, used during distcheck. You
# probably won't fill this in but rather it's filled in by the `./configure`
# script. Useful for debugging.
configure-args = ['--release-channel=nightly', '--enable-llvm-enzyme', '--enable-llvm-assertions', '--enable-option-checking', '--disable-docs', '--set', 'llvm.download-ci-llvm=true']

[install]

[rust]

# The "channel" for the Rust build to produce. The stable/beta channels only
# allow using stable features, whereas the nightly and dev channels allow using
# nightly features.
#
# You can set the channel to "auto-detect" to load the channel name from `src/ci/channel`.
#
# If using tarball sources, default value is "auto-detect", otherwise, it's "dev".
channel = 'nightly'

[dist]

[target.aarch64-apple-darwin]

Operating system

Mac OS 15.4.1 (M2 Macbook Pro).

HEAD

b2ee1b3

Additional context

Build Log

build_output.txt

[ZuseZ4]

Thanks for investigating and trying the old command as well. To be sure, did you rm the whole build folder in between (or use different rust clones)? I made the experience that sometimes bootstrap doesn't properly delete or update old artifacts. If the error remains, can you also try the same build with llvm.enzyme set to false? Just as a smoke test to be sure it's the fault of autodiff, since at least building Enzyme (our autodiff backend) seems to work fine based on your logs.
Unfortunately I won't have access to my own Mac for the next two weeks, so I can't reproduce it atm.

[kevinyamauchi]

Thanks for the reply, @ZuseZ4 !

To be sure, did you rm the whole build folder in between (or use different rust clones)? I made the experience that sometimes bootstrap doesn't properly delete or update old artifacts.

Yeah for each attempt, I deleted the whole rust folder and and cloned a fresh copy of the repo.

If the error remains, can you also try the same build with llvm.enzyme set to false?

Good suggestion! I tried building with llvm.enzyme set to false using the commands below. The build completed successfully (logs below) and the test suite passed (./X test). I also tested the build instructions in the guide on a machine with Ubuntu and an Intel processor and that worked (including the autodiff tests). I suppose that means the issue is indeed related to building on the Mac.

git clone git@github.com:rust-lang/rust
cd rust
./configure --release-channel=nightly  --enable-llvm-assertions --enable-option-checking --disable-docs --set llvm.download-ci-llvm=true
RUST_BACKTRACE=1 ./x build -v --stage 1 library | tee build_output.txt

lastest commit hash: 11d2046fe9962720558cb15f72541f7fd170dec9

bootstrap.toml

# Use different pre-set defaults than the global defaults.
#
# See `src/bootstrap/defaults` for more information.
# Note that this has no default value (x.py uses the defaults in `bootstrap.example.toml`).
profile = 'dist'

[llvm]

# Currently, we only support this when building LLVM for the build triple.
#
# Note that many of the LLVM options are not currently supported for
# downloading. Currently only the "assertions" option can be toggled.
download-ci-llvm = true

# Indicates whether the LLVM assertions are enabled or not
# NOTE: When assertions are disabled, bugs in the integration between rustc and LLVM can lead to
# unsoundness (segfaults, etc.) in the rustc process itself, not just in the generated code.
assertions = true

[gcc]

[build]

# Whether to build documentation by default. If false, rustdoc and
# friends will still be compiled but they will not be used to generate any
# documentation.
#
# You can still build documentation when this is disabled by explicitly passing paths,
# e.g. `x doc library`.
docs = false

# Arguments passed to the `./configure` script, used during distcheck. You
# probably won't fill this in but rather it's filled in by the `./configure`
# script. Useful for debugging.
configure-args = ['--release-channel=nightly', '--enable-llvm-assertions', '--enable-option-checking', '--disable-docs', '--set', 'llvm.download-ci-llvm=true']

[install]

[rust]

# The "channel" for the Rust build to produce. The stable/beta channels only
# allow using stable features, whereas the nightly and dev channels allow using
# nightly features.
#
# You can set the channel to "auto-detect" to load the channel name from `src/ci/channel`.
#
# If using tarball sources, default value is "auto-detect", otherwise, it's "dev".
channel = 'nightly'

[dist]

[target.aarch64-apple-darwin]

build log

build_output.txt

Unfortunately I won't have access to my own Mac for the next two weeks, so I can't reproduce it atm.

No worries. As mentioned above, I got it working on my linux machine, so I'm not totally blocked. However, it would be nice if I could get it working on my Mac 😉

[ZuseZ4]

Glad you have at least linux working.
If you have time to look into it further, you should probably try to add a dbg! statement to those two lines:

rustc -Cllvm-args="..." with: Unknown command line argument '-mergefunc-use-aliases'.  Try: 'rustc -Cllvm-args="..." with --help'
[...]
Created at: src/bootstrap/src/core/builder/mod.rs:1614:23
Executed at: src/bootstrap/src/core/build_steps/compile.rs:538:26

Also, looking at the (incomplete) rustc invocation failing with a lot of llvm args, my assumption is that the formating or something with one of the earlier commands is wrong, and then all following flags are not recognized. However, tha'ts hard to tell without seeing the full command. In autodiff we don't change the args mentioned in your error log, but I saw that the only usage of mergefunc-use-aliases within rustc is in
compiler/rustc_codegen_llvm/src/llvm_util.rs. If you add another dbg statement at the end of the configure_llvm function to print the command, we might now more.
If you get spammed by too much output you might want to look into the Tracing infra, there is a rustc-dev-guide entry about it. If you post the output of those we might be able to figure out what's wrong.

[ZuseZ4]

fwiw github.com/rust-lang/enzyme/pull/30 macos ci also fails here.

[kevinyamauchi]

Thanks for the additional debugging tips, @ZuseZ4 . As suggested, I added some debugging statements around where the build is failing. I made the changes below. Based on the output, I can see the conditionals that are leading to the unrecognized arguments in configure_llvm(), but I am not at all familiar with the build system to know why they are triggered. I am happy to keep debugging if you have additional tips or ideas 😄

src/bootstrap/src/core/builder/mod.rs:1613

    pub fn rustc_cmd(&self, compiler: Compiler) -> BootstrapCommand {
        let mut cmd = command(self.rustc(compiler));
        dbg!(&cmd);
        self.add_rustc_lib_path(compiler, &mut cmd);
        dbg!(&cmd);
        cmd
    }

src/bootstrap/src/core/build_steps/compile.rs:531

    if target.contains("apple") && !builder.config.dry_run() {
        // Query rustc for the deployment target, and the associated env var.
        // The env var is one of the standard `*_DEPLOYMENT_TARGET` vars, i.e.
        // `MACOSX_DEPLOYMENT_TARGET`, `IPHONEOS_DEPLOYMENT_TARGET`, etc.
        let mut cmd = builder.rustc_cmd(cargo.compiler());
        cmd.arg("--target").arg(target.rustc_target_arg());
        cmd.arg("--print=deployment-target");
        dbg!(&cmd);
        dbg!(&builder);
        let output = cmd.run_capture_stdout(builder).stdout();
        dbg!(&output);

/compiler/rustc_codegen_llvm/src/llvm_util.rs:73 (to print the string argument names in configure_llvm())

        // This adds the given argument to LLVM. Unless `force` is true
        // user specified arguments are *not* overridden.
        let mut add = |arg: &str, force: bool| {
            if force || !user_specified_args.contains(llvm_arg_to_arg_name(arg)) {
                dbg!(arg);
                let s = CString::new(arg).unwrap();
                llvm_args.push(s.as_ptr());
                llvm_c_strs.push(s);
            }
        };

/compiler/rustc_codegen_llvm/src/llvm_util.rs:147

    rustc_llvm::initialize_available_targets();

    dbg!(&llvm_args);

    unsafe { llvm::LLVMRustSetLLVMOptions(llvm_args.len() as c_int, llvm_args.as_ptr()) };

This yields the following output (deleted the source and re-cloned):

[src/bootstrap/src/core/builder/mod.rs:1615:9] &cmd = "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" (failure_mode=Exit)
[src/bootstrap/src/core/builder/mod.rs:1617:9] &cmd = DYLD_LIBRARY_PATH="/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/lib" "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" (failure_mode=Exit)
[src/bootstrap/src/core/build_steps/compile.rs:538:9] &cmd = DYLD_LIBRARY_PATH="/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/lib" "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" "--target" "aarch64-apple-darwin" "--print=deployment-target" (failure_mode=Exit)
running: DYLD_LIBRARY_PATH="/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/lib" "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" "--target" "aarch64-apple-darwin" "--print=deployment-target" (failure_mode=Exit) (created at src/bootstrap/src/core/builder/mod.rs:1614:23, executed at src/bootstrap/src/core/build_steps/compile.rs:539:26)
[compiler/rustc_codegen_llvm/src/llvm_util.rs:77:17] arg = "rustc -Cllvm-args=\"...\" with"
[compiler/rustc_codegen_llvm/src/llvm_util.rs:77:17] arg = "-generate-arange-section"
[compiler/rustc_codegen_llvm/src/llvm_util.rs:77:17] arg = "-mergefunc-use-aliases"
[compiler/rustc_codegen_llvm/src/llvm_util.rs:77:17] arg = "-preserve-alignment-assumptions-during-inlining=false"
[compiler/rustc_codegen_llvm/src/llvm_util.rs:77:17] arg = "-import-cold-multiplier=0.1"
[compiler/rustc_codegen_llvm/src/llvm_util.rs:149:5] &llvm_args = [
    0x0000600000d10060,
    0x0000600000d10000,
    0x0000600000d102a0,
    0x00006000018143c0,
    0x0000600000d102c0,
]
rustc -Cllvm-args="..." with: Unknown command line argument '-mergefunc-use-aliases'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--riscv-no-aliases'?
rustc -Cllvm-args="..." with: Unknown command line argument '-preserve-alignment-assumptions-during-inlining=false'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--mips-align-constant-islands=false'?
rustc -Cllvm-args="..." with: Unknown command line argument '-import-cold-multiplier=0.1'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--ignore-xcoff-visibility=0.1'?
Command `DYLD_LIBRARY_PATH="/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/lib" "/Users/kyamauch/Documents/rust_source/rust/build/aarch64-apple-darwin/stage1/bin/rustc" "--target" "aarch64-apple-darwin" "--print=deployment-target" (failure_mode=Exit)` failed with exit code 1
Created at: src/bootstrap/src/core/builder/mod.rs:1614:23
Executed at: src/bootstrap/src/core/build_steps/compile.rs:539:26

build log

build_output.txt

fwiw github.com/rust-lang/enzyme#30 macos ci also fails here.

Good to know it's not just me 😉

[ZuseZ4]

Hmm, if you say that you alao have a working linux system, could you use your mac for some automated bisection?
https://rustc-dev-guide.rust-lang.org/compiler-debugging.html?highlight=bisecting#narrowing-bisecting-regressions

I think starting with a 3 month old checkout should be good. Just configure it to always build llvm from source.
I never had to use it, but I heared good things about it

[kevinyamauchi]

Thanks for the pointer to the cargo-bisect tool. Admittedly, I didn't try very hard, but I couldn't figure out how to get it to work with building from source, so I started with normal git bisect. I went back to 5210c50 (Aug. 1, 2025) and the build still failed with the same error. I used the "old" config using LLVM built from source. I wrote the commands I used and the error I received below.

Before I go back further, I just wanted to check in to see if this is what you had in mind. My concern is that given that on current main rustc builds without the autodiff and Enzyme linking and that the Enzyme CI you linked above is also failing on Mac, this issue might be somehow related to Enzyme. I'm not sure how the linked version of Enzyme is specified in the build, so I am not sure if going back in rust commits will actually pull down different versions of Enzyme.

This is what I tried:

# download a fresh rust source
git clone git@github.com:rust-lang/rust
cd rust

# start git bisect and mark the current had as bad
git bisect start
git bisect bad

# checkout an old commit from Aug. 1, 2025
git checkout 5210c501bc7320d7cc5e84f77d2c7ca5fca9a2e0

# configure and build
./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
 ./x build  --stage 1 library

I received the same error as before:

error message

Building stage1 tool lld-wrapper (aarch64-apple-darwin)
   Compiling lld-wrapper v0.1.0 (/Users/kyamauch/Documents/rust_source/rust/src/tools/lld-wrapper)
    Finished `release` profile [optimized] target(s) in 0.87s
Building stage1 tool wasm-component-ld (aarch64-apple-darwin)
  Downloaded wasi-preview1-component-adapter-provider v34.0.2
  Downloaded wasm-encoder v0.235.0
  Downloaded wat v1.235.0
  Downloaded wasm-metadata v0.234.0
  Downloaded clap_derive v4.5.41
  Downloaded wasm-component-ld v0.5.15
  Downloaded anyhow v1.0.98
  Downloaded clap v4.5.41
  Downloaded wasm-encoder v0.234.0
  Downloaded anstream v0.6.19
  Downloaded anstyle-query v1.1.3
  Downloaded wit-parser v0.234.0
  Downloaded wast v235.0.0
  Downloaded clap_builder v4.5.41
  Downloaded wit-component v0.234.0
  Downloaded wasmparser v0.235.0
  Downloaded 16 crates (1.4MiB) in 0.45s
   Compiling serde v1.0.219
   Compiling proc-macro2 v1.0.95
   Compiling unicode-ident v1.0.18
   Compiling semver v1.0.26
   Compiling bitflags v2.9.1
   Compiling foldhash v0.1.5
   Compiling leb128fmt v0.1.0
   Compiling equivalent v1.0.2
   Compiling libc v0.2.174
   Compiling wasmparser v0.234.0
   Compiling anyhow v1.0.98
   Compiling memchr v2.7.5
   Compiling serde_json v1.0.141
   Compiling wasm-encoder v0.234.0
   Compiling wasm-encoder v0.235.0
   Compiling utf8parse v0.2.2
   Compiling anstyle-parse v0.2.7
   Compiling anstyle-query v1.1.3
   Compiling quote v1.0.40
   Compiling syn v2.0.104
   Compiling anstyle v1.0.11
   Compiling getrandom v0.3.3
   Compiling colorchoice v1.0.4
   Compiling ryu v1.0.20
   Compiling is_terminal_polyfill v1.70.1
   Compiling rustix v1.0.8
   Compiling itoa v1.0.15
   Compiling anstream v0.6.19
   Compiling errno v0.3.13
   Compiling clap_lex v0.7.5
   Compiling unicode-xid v0.2.6
   Compiling cfg-if v1.0.1
   Compiling id-arena v2.2.1
   Compiling bumpalo v3.19.0
   Compiling strsim v0.11.1
   Compiling unicode-width v0.2.1
   Compiling heck v0.5.0
   Compiling log v0.4.27
   Compiling clap_builder v4.5.41
   Compiling fastrand v2.3.0
   Compiling once_cell v1.21.3
   Compiling wasi-preview1-component-adapter-provider v34.0.2
   Compiling lexopt v0.3.1
   Compiling wast v235.0.0
   Compiling hashbrown v0.15.4
   Compiling indexmap v2.10.0
   Compiling serde_derive v1.0.219
   Compiling clap_derive v4.5.41
   Compiling tempfile v3.20.0
   Compiling clap v4.5.41
   Compiling wat v1.235.0
   Compiling wit-parser v0.234.0
   Compiling wasm-metadata v0.234.0
   Compiling wit-component v0.234.0
   Compiling wasm-component-ld v0.5.15
   Compiling wasm-component-ld-wrapper v0.1.0 (/Users/kyamauch/Documents/rust_source/rust/src/tools/wasm-component-ld)
    Finished `release` profile [optimized] target(s) in 19.71s
Building stage1 tool llvm-bitcode-linker (aarch64-apple-darwin)
   Compiling once_cell v1.21.3
   Compiling lazy_static v1.5.0
   Compiling overload v0.1.1
   Compiling log v0.4.27
   Compiling syn v2.0.104
   Compiling thiserror v1.0.69
   Compiling thread_local v1.1.9
   Compiling smallvec v1.15.1
   Compiling nu-ansi-term v0.46.0
   Compiling sharded-slab v0.1.7
   Compiling tracing-core v0.1.30
   Compiling pin-project-lite v0.2.16
   Compiling tracing-log v0.2.0
   Compiling tracing-subscriber v0.3.18
   Compiling tracing-attributes v0.1.30
   Compiling clap_derive v4.5.41
   Compiling thiserror-impl v1.0.69
   Compiling tracing v0.1.37
   Compiling clap v4.5.41
   Compiling llvm-bitcode-linker v0.0.1 (/Users/kyamauch/Documents/rust_source/rust/src/tools/llvm-bitcode-linker)
    Finished `release` profile [optimized] target(s) in 3.51s
rustc -Cllvm-args="..." with: Unknown command line argument '-generate-arange-section'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--enzyme-coalese'?
rustc -Cllvm-args="..." with: Unknown command line argument '-mergefunc-use-aliases'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--enzyme-noalias'?
rustc -Cllvm-args="..." with: Unknown command line argument '-preserve-alignment-assumptions-during-inlining=false'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--enzyme-assume-unknown-nofree=false'?
rustc -Cllvm-args="..." with: Unknown command line argument '-import-cold-multiplier=0.1'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--print-all-options=0.1'?
Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 0:33:51

[ZuseZ4]

That's interesting. I've build on a mac since, but maybe my checkout was older, or there is a difference between m2 and m4 here.
I'd try an older commit, I don't know git bisect, but your commands look right.

One thing that you could try independently on master is to build without the llvm-link-shared flag. It's less tested, but I did add support for it at some point.

[kevinyamauchi]

Thanks for the feedback, @ZuseZ4 ! I'm slowly making progress with git bisect. It's a bit slow because the build takes ~30 mins. on my Mac. However, I've started to narrow in on the breaking commit:

• most recent good commit found: d5a471c (July 17)
• earliest bad commit found: 0570fb4 (July 27)

There are ~9 iterations left to narrow in on the breaking commit. I will keep updating here as I make progress.