bootstrap should not use `rustc --version` to detect the build triple on on Mac Apple Silicon M1 arm64

[HarrisonMc555]

TL;DR

I understand that Apple Silicon/M1/Arm is not supported as a tier 1 target. However, I was hoping that I would be able to compile the Rust compiler with Rosetta support. However, I could not do so.

Details

I did a fresh clone of the rust repository and tried to compile it by running ./x.py. However, it gave me the following errors:

  = note: ld: warning: ignoring file /opt/homebrew/Cellar/xz/5.2.7/lib/liblzma.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
          Undefined symbols for architecture x86_64:
            "_lzma_code", referenced from:
                xz2::stream::Stream::process::h73782f45eb644e15 in libxz2-35bd6285e11c4f9a.rlib(xz2-35bd6285e11c4f9a.xz2.e97199be-cgu.2.rcgu.o)
            "_lzma_end", referenced from:
                _$LT$xz2..stream..Stream$u20$as$u20$core..ops..drop..Drop$GT$::drop::h9a9d62c278838afd in libxz2-35bd6285e11c4f9a.rlib(xz2-35bd6285e11c4f9a.xz2.e97199be-cgu.2.rcgu.o)
            "_lzma_stream_decoder", referenced from:
                xz2::stream::Stream::new_stream_decoder::hfed3d487f93f4786 in libxz2-35bd6285e11c4f9a.rlib(xz2-35bd6285e11c4f9a.xz2.e97199be-cgu.2.rcgu.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

I expected to see this happen: I expected to be able to compile the Rust compiler on my machine.

Instead, this happened: I could not compile the Rust compiler on my machine.

I can see the warning that "building for macOS-x86_64 but attempting to link with file built for macOS-arm64". However, I don't know how to change that. I'm also a bit confused by the difference in reporting from the rust tools themselves vs. the file command. The tools themselves report "x86_64", but the file command reports "arm64".

$ rustc --version --verbose
rustc 1.64.0 (a55dd71d5 2022-09-19)
binary: rustc
commit-hash: a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52
commit-date: 2022-09-19
host: x86_64-apple-darwin
release: 1.64.0
LLVM version: 14.0.6

$ file $(which rustc)
/Users/harrisonmccullough/.cargo/bin/rustc: Mach-O 64-bit executable arm64

$ cargo --version --verbose
cargo 1.64.0 (387270bc7 2022-09-16)
release: 1.64.0
commit-hash: 387270bc7f446d17869c7f208207c73231d6a252
commit-date: 2022-09-16
host: x86_64-apple-darwin
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.79.1 (sys:0.4.55+curl-7.83.1 system ssl:(SecureTransport) LibreSSL/3.3.6)
os: Mac OS 12.5.1 [64-bit]

$ file $(which cargo)
/Users/harrisonmccullough/.cargo/bin/cargo: Mach-O 64-bit executable arm64

Meta

$ rustc --version --verbose
rustc 1.64.0 (a55dd71d5 2022-09-19)
binary: rustc
commit-hash: a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52
commit-date: 2022-09-19
host: x86_64-apple-darwin
release: 1.64.0
LLVM version: 14.0.6

$ uname -a
Darwin Harrison-MacBook-Pro 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:28:23 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T6000 arm64 arm Darwin

$ git log --format=%H -n1
736c675d2ab65bcde6554e1b73340c2dbc27c85a

[workingjubilee]

You can manage to compile Rust in this manner, but you must specifically configure it for cross-compiling. Rosetta2 produces these results because the machine is trying to say it is both an aarch64 and x86-64 machine to different things, so bootstrap does not successfully navigate this. You must hold its hand.

However, this is all irrelevant. It having "only" Tier 2 support does not prevent us from distributing an aarch64-apple-darwin host compiler toolchain, including most optional components through Rustup, so please use that instead of... suffering through these shenanigans.

[chenyukang]

@workingjubilee , oops, I plan to get a Macbook M2, it seems it's not suitable for Rust compiler development?

[HarrisonMc555]

You can manage to compile Rust in this manner, but you must specifically configure it for cross-compiling. Rosetta2 produces these results because the machine is trying to say it is both an aarch64 and x86-64 machine to different things, so bootstrap does not successfully navigate this. You must hold its hand.

That makes sense, thanks! I suspected something like this might be going on. Do you have any suggestions for how to "hold its hand"? Or what resources might be helpful?

However, this is all irrelevant. It having "only" Tier 2 support does not prevent us from distributing an aarch64-apple-darwin host compiler toolchain, including most optional components through Rustup, so please use that instead of... suffering through these shenanigans.

My apologies; I should have made it clear that I have the normal Rust toolchain installed via the normal methods and it has all worked great! I can compile my own Rust programs and install tools via cargo install and everything like that. I wanted to get x.py to work because I was interested in contributing to the compiler itself.

I did a fair amount of Googling to see if there were other places where people talked about contributing to the Rust compiler using an Apple Silicon/M1/etc. machine and didn't find anything. That's why I posted here 😄 Any pointers would be appreciated, for my sake and for the sake of anybody else who wants to do the same and finds this thread.

[workingjubilee]

I believe the simplest way will be to edit the config.toml. It has various options, which you probably want to set, like

# Build triple for the original snapshot compiler. This must be a compiler that
# nightlies are already produced for. The current platform must be able to run
# binaries of this build triple and the nightly will be used to bootstrap the
# first compiler.
#
# Defaults to platform where `x.py` is run.
build = "aarch64-apple-darwin" # because Rosetta confuses bootstrap

and maybe

# Instead of downloading the src/stage0.json version of Cargo specified, use
# this Cargo binary instead to build all Rust code
cargo = "/path/to/cargo" # bypassing downloads that might get the wrong compiler

You will want to peruse the config.toml.example for the full list of options.

It seems that https://github.com/rust-lang/rustc-dev-guide needs a PR listing explicit instructions for how to get aarch64-apple-darwin building, or we need a PR here to bootstrap to get aarch64-apple-darwin building "automatically". Either one.

[luqmana]

@workingjubilee is right, just because the aarch64-apple-darwin target isn't tier 1 doesn't mean you can't build it. I build rust on an m1 mac every so often without any thing special.

My config.toml

# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
# profile = "codegen"

changelog-seen = 2

Usually I just want a stage1 build which I get like so:

$ ./x.py build -i library/std
[...]
    Finished release [optimized] target(s) in 0.13s
Copying stage1 std from stage1 (aarch64-apple-darwin -> aarch64-apple-darwin / aarch64-apple-darwin)
Build completed successfully in ...
$ rustup toolchain link stage1 build/aarch64-apple-darwin/stage1
$ rustc +stage1 -vV
rustc 1.67.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: aarch64-apple-darwin
release: 1.67.0-dev
LLVM version: 15.0.4

I also haven't enabled Rosetta on this device so this is all native arm64.

[workingjubilee]

In that case I suppose the advisory of note is to suggest people try to actively not use Rosetta when compiling rustc.

[luqmana]

I haven't given it a try, but I wonder if setting the host/build triples to aarch64-apple-darwin in config.toml would be enough. In any case, seems like something the build tooling should handle automatically.

EDIT: Ok, looking at bootstrap it does seem like it should detect things properly. But if there's an existing rustc installed (e.g. from rustup) it will use whatever host triple that was configured with, otherwise it does uname-based auto detection. Running x.py with the verbose flag should output either "detected default triple ... from pre-installed rustc" or "falling back to auto-detect", respectively.

[luqmana]

@HarrisonMc555 Could you check the output of file $(rustc --print sysroot)/bin/rustc? The rustc in ~/.cargo/bin is just a wrapper. It's possible your default toolchain is the x86_64 version, hence the mismatch.

[HarrisonMc555]

Thanks for your help, everyone! I made these changes to config.toml

build = "aarch64-apple-darwin" # because Rosetta confuses bootstrap
# Instead of downloading the src/stage0.json version of Cargo specified, use
# this Cargo binary instead to build all Rust code
cargo = "/path/to/cargo" # bypassing downloausds that might get the wrong compiler

And that worked! Thanks @workingjubilee!

---

@luqmana here's what you asked for:

$ file $(rustc --print sysroot)/bin/rustc
/Users/harrisonmccullough/.rustup/toolchains/stable-x86_64-apple-darwin/bin/rustc: Mach-O 64-bit executable x86_64

$ file $(which rustc)
/Users/harrisonmccullough/.cargo/bin/rustc: Mach-O 64-bit executable arm64

So that's weird. Where is my .cargo/bin/rustc coming from? And why does it get to be an arm64 executable, while the stable version is x86_64?

...

So I think I may have an idea about what happened. Previously, I had an Intel x86_64 MacBook. I then got a new M1 Apple Silicon MacBook. I used Apple's Time Machine to migrate everything from my old computer to my new computer. I had done Rust work on my previous laptop. So maybe the default stable toolchain was set to x86_64, which was correct for the previous computer, and never got updated when I migrated to the new computer? Does that seem plausible?

[luqmana]

~/.cargo/bin/rustc (and cargo there) is from rustup and is just a wrapper that enables the rustc +nightly ... or cargo +stable ... conveniences.

[jyn514]

hmm. I remember @DebugSteven hit this same issue a while ago. I wonder if - on MacOS only - we should avoid inferring the build triple from rustup. I added it originally to detect windows-gnu vs windows-msvc, I hadn't considered Rosetta.