Possibly non-isolated build on macOS

[uwu-420]

Hi, thanks for maintaining this wonderful project :)

If I understood correctly, one of this project's goals is to isolate as good as possible for builds on macOS, which is why a fixed version of LLVM is pulled in during build time.

I wanted to try making specific builds for the various Apple M processors by adding e.g. -mcpu=apple-m3 to the target_cflags and target_ldflags in the aarch64-apple-darwin target in targets.yml. (Btw, is that a cursed thing to try?)

For -mcpu=apple-m2 everything works nicely, but for -mcpu=apple-m3 (or m4) the build fails. I couldn't find anything informative in the logs that would point to the issue, so maybe someone can reproduce this?

But that kind of fits a suspicion I have, namely that the LLVM from XCode is notably involved in the build process (which it shouldn't?) and it does not know about M3 or M4 processors, which is why the build fails. When I run clang --version I get Apple clang version 16.0.0 (clang-1600.0.26.3) and I'm using macOS 15.1.1 (24B91) on a M3 MacBook so the LLVM version indeed does not know about M3 and M4. See https://releases.llvm.org/ and https://en.wikipedia.org/wiki/Apple_M3 for the according dates.

Don't know how relevant this is to you but maybe this points to some unintended non-isolation.

[uwu-420]

Couldn't confirm it yet, but maybe it's due to Apple's /usr/bin/gcc shim wrapper that basically redirects to Apple's clang and there might be something in the build process that accidentally always uses gcc?

[zanieb]

As a minor note, we should not be building with Apple Clang. We use a separate LLVM toolchain

python-build-standalone/pythonbuild/downloads.py

Lines 194 to 206 in 1ab3596

	"llvm-aarch64-macos": {
	"url": "https://github.com/indygreg/toolchain-tools/releases/download/toolchain-bootstrap%2F20241231/llvm-19.1.6+20241231-aarch64-apple-darwin.tar.zst",
	"size": 143884729,
	"sha256": "c0252c7ebe0b20125fe592a12d43a693fcab0bfa3bec9def6b96b97087e0f765",
	"version": "19.1.6+20241231",
	},
	# Remember to update LLVM_URL in src/release.rs whenever upgrading.
	"llvm-x86_64-macos": {
	"url": "https://github.com/indygreg/toolchain-tools/releases/download/toolchain-bootstrap%2F20241231/llvm-19.1.6+20241231-x86_64-apple-darwin.tar.zst",
	"size": 143113277,
	"sha256": "e990dea9ff89e4202d358b477a43f2bd128b244926534b7c8bb29feeaada994e",
	"version": "19.1.6+20241231",
	},

[zanieb]

Can you share a simple patch that you're using to reproduce the failure?

[uwu-420]

Yeah that's what I mean, it should be independent of Apple Clang, but I suspect it isn't.

Here is the patch with which the compilation will not work

diff --git a/cpython-unix/targets.yml b/cpython-unix/targets.yml
index d328589..35e5f33 100644
--- a/cpython-unix/targets.yml
+++ b/cpython-unix/targets.yml
@@ -74,6 +74,7 @@ aarch64-apple-darwin:
     - '-arch'
     - 'arm64'
     - '-mmacosx-version-min=11.0'
+    - '-mcpu=apple-m3'
     # Suppress extremely verbose warnings we see with LLVM 10.
     - '-Wno-nullability-completeness'
     - '-Wno-expansion-to-defined'

I run the compilation with python3 build-macos.py --target-triple="aarch64-apple-darwin" --python="cpython-3.13" --options="pgo+lto".
And most interestingly, when I use -mcpu=apple-m2 instead of -mcpu=apple-m3 it works!
I hope my initial explanation of the hypothesis in this issue was clear, but I can try to rephrase if it wasn't :)

(I know that using this flag is really pointless, but I was curious and tried for the lulz. This is about the build process maybe using Apple Clang although it shouldn't.)

[zanieb]

Let me see if I can reproduce first. I'm not sure how we could possibly be using the wrong toolchain, lots of things would be broken.

[zanieb]

I get

libffi> configure: error: C++ preprocessor "/lib/cpp" fails sanity check

Does that sound right to you?

[zanieb]

Then, picking at the config output

configure:5431: checking whether the compiler supports GNU C++
configure:5451: g++ -c  -arch arm64 -mmacosx-version-min=11.0 -mcpu=apple-m3 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -fvisibility=hidden -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fPIC conftest.cpp >&
5
clang++: error: unsupported argument 'apple-m3' to option '-mcpu='

[zanieb]

Which I can indeed reproduce on my local Clang, but that's also 19.1.6

❯ g++ -c -arch arm64 -mmacosx-version-min=11.0 -mcpu=apple-m3 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -fvisibility=hidden -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fPIC foo.cpp
clang++: error: unsupported argument 'apple-m3' to option '-mcpu='
❯ clang++ --version
Homebrew clang version 19.1.6
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.6/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg

So I guess I don't see this error as evidence that we're using the an old Apple toolchain.

[uwu-420]

Wait, is your g++ also the homebrew one here? Because for me it works.
Edit: I mean that g++ is just Apple's alias for it's own clang++ right? So this one I wouldn't expect to understand that flag anyway.

➜  python-build-standalone git:(main) ✗ /opt/homebrew/Cellar/llvm/19.1.7/bin/clang++ --version                    
Homebrew clang version 19.1.7
Target: arm64-apple-darwin24.3.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.7/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin24.cfg
➜  python-build-standalone git:(main) ✗ /opt/homebrew/Cellar/llvm/19.1.7/bin/clang++ -mcpu=apple-m3 foo.cpp -o foo
➜  python-build-standalone git:(main) ✗ ./foo 
Hello, World!
➜  python-build-standalone git:(main) ✗ /opt/homebrew/Cellar/llvm/19.1.7/bin/clang++ -mcpu=apple-m5 foo.cpp -o foo
clang++: error: unsupported argument 'apple-m5' to option '-mcpu='

Am I missing something?

[uwu-420]

I get

libffi> configure: error: C++ preprocessor "/lib/cpp" fails sanity check

Does that sound right to you?

Yeah, this one I also get.

Then, picking at the config output

configure:5431: checking whether the compiler supports GNU C++
configure:5451: g++ -c  -arch arm64 -mmacosx-version-min=11.0 -mcpu=apple-m3 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -fvisibility=hidden -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fPIC conftest.cpp >&
5
clang++: error: unsupported argument 'apple-m3' to option '-mcpu='

This one I don't 🤔

[zanieb]

I thought the pitch was that new versions of clang should understand that flag?

[zanieb]

This one I don't

I pulled that from the failed libffi config.log output, just fyi.

[uwu-420]

I thought the pitch was that new versions of clang should understand that flag?

Yeah and they do. I think you are using Apple clang by invoking g++ but to use your recent homebrew clang you should use clang++ right? Then you'd see that the flag works fine. While it doesn't work in the build process in this repo.

What does g++ --version give you?

[zanieb]

Oh! I see, I'm an idiot :)

❯ clang++ -c -arch arm64 -mmacosx-version-min=11.0 -mcpu=apple-m3 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -fvisibility=hidden -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fPIC foo.cpp
❯ g++ --version
Apple clang version 16.0.0 (clang-1600.0.26.6)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Thanks for your patience. Sounds like this could indeed be a bug.