[modules] -print-file-name=libc++.modules.json does not work in mac m1

[ChuanqiXu9]

I received a private report that -print-file-name=libc++.modules.json doesn't work well in mac m1. Maybe this is related to the installation path of libc++ in MacOS.

Verify needed.

[llvmbot]

@llvm/issue-subscribers-clang-modules

Author: Chuanqi Xu (ChuanqiXu9)

I received a private report that `-print-file-name=libc++.modules.json` doesn't work well in mac. Maybe this is related to the installation path of libc++ in MacOS.

Verify needed.

[ChuanqiXu9]

Also it looks like -print-file-name= doesn't work well for other files in mac m1. According to my experiment, in linux, it works well to return the path as:

/disk2/workspace.xuchuanqi/llvm-project-for-work/build_clang_libcxx/installed/bin/../lib/x86_64-unknown-linux-gnu/libc++.modules.json

but in mac m1, the libc++ modules json file is installed to

<install-dir>/lib/libc++.modules.json

then x86_64-unknown-linux-gnu becomes a significant change. I am not sure if this is the root of the cause.

[llvmbot]

@llvm/issue-subscribers-clang-driver

Author: Chuanqi Xu (ChuanqiXu9)

I received a private report that `-print-file-name=libc++.modules.json` doesn't work well in mac m1. Maybe this is related to the installation path of libc++ in MacOS.

Verify needed.

[ChuanqiXu9]

I guess we need help from Apple's people @Bigcheese @jansvoboda11 @ldionne

[billhoffman]

Not sure if this helps but:

% clang++ -print-file-name=libc++.modules.json
libc++.modules.json
% clang++ -print-file-name=../../libc++.modules.json
/Users/hoffman/Work/llvm/llvm-inst/lib/clang/20/../../libc++.modules.json

clang++ --version
clang version 20.0.0git (git@github.com:llvm/llvm-project.git d5dd7d2)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Users/hoffman/Work/llvm/llvm-inst/bin

[ldionne]

This seems similar to #98325.

@Xazax-hun Any appetite for this one?

[ChuanqiXu9]

Some additional information about the codes, hope this helps:

The -print-file-name was implemented in Driver::GetFilePath from clang/lib/Driver/Driver.cpp. The key value here are TC.getLibraryPaths() and TC.getFilePaths(). Their values are initialized in ToolChain::ToolChain from clang/lib/Driver/ToolChain.cpp. Note that in the comment of ToolChain::getRuntimePath() it says:

// Darwin and AIX does not use per-target runtime directory.

and in ToolChain::getStdlibPath(), it will add the target triple automatically. So I feel it is broken in mac since day 1. A simple fix may be not to add the target triple in ToolChain::getStdlibPath().

[billhoffman]

The following patch seems to work for me:

index efe398dd531d..78d06a68bf0d 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -192,8 +192,12 @@ std::string Driver::GetResourcesPath(StringRef BinaryPath) {
     P = llvm::sys::path::parent_path(Dir);
     // This search path is also created in the COFF driver of lld, so any
     // changes here also needs to happen in lld/COFF/Driver.cpp
-    llvm::sys::path::append(P, CLANG_INSTALL_LIBDIR_BASENAME, "clang",
-                            CLANG_VERSION_MAJOR_STRING);
+    llvm::sys::path::append(P, CLANG_INSTALL_LIBDIR_BASENAME);
+    // Darwin and AIX does not use per-target runtime directory.
+    if (!llvm::Triple(llvm::sys::getProcessTriple()).isOSDarwin()) {
+      llvm::sys::path::append(P, "clang",
+                              CLANG_VERSION_MAJOR_STRING);
+    }
   }

[mathstuf]

• // Darwin and AIX does not use per-target runtime directory.
• if (!llvm::Triple(llvm::sys::getProcessTriple()).isOSDarwin()) {

A minor nit: the comment mentions AIX but the code doesn't.

[billhoffman]

• // Darwin and AIX does not use per-target runtime directory.
• if (!llvm::Triple(llvm::sys::getProcessTriple()).isOSDarwin()) {

A minor nit: the comment mentions AIX but the code doesn't.

Yes, I think the code should be updated to do the same for AIX, this comment was copied from another location. Wanted to make sure this is the right approach before cleaning it up. If this looks good to @ChuanqiXu9 and others, I can make a PR with cleaned up code.

[ChuanqiXu9]

It looks good to me. But I think we need double check from apple forks or Driver owners @jansvoboda11 @Bigcheese @MaskRay

[billhoffman]

Any way to move this along? Should I just create a PR?

[ldionne]

I think that would be the quickest way to ensure this gets addressed, since you already have the patch.

[ChuanqiXu9]

Yes, +1

[billhoffman]

Oh dear... I created a fork and went to compile and now I am getting a new error. This is from the bootstrapped compiler not being able to find stdbool.h. I think it is some update of xcode that has caused this. Anyone seen this?
/Users/hoffman/Work/llvm/llvm-project/libunwind/src/UnwindLevel1.c:23:10: fatal error: 'stdbool.h' file not found

[ChuanqiXu9]

I didn't see it. But I think we didn't force contributors to bootstrap their build as far as I know. So maybe we can workaround that.

[billhoffman]

I created an issue: #112151 (comment)

[billhoffman]

@ChuanqiXu9 do you know anything about how the default include paths get constructed for the clang that is in the build tree I think I found the issue. See my last comment in the issue I created.

[ChuanqiXu9]

clang/lib/Driver/ToolChain.cpp

I think it is ToolChain::getStdlibIncludePath()

[billhoffman]

OK, the patch is what is breaking the build. So, my patch does not work. I think perhaps the location of libc++.modules.json should be changed on the mac to match where it is on linux, so that this works.

[billhoffman]

@petrhosek can you look at this? I think it is related to the install location of libc++.modules.json. Using git blame I can see you put this code in: if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND NOT APPLE) I am guessing the NOT APPLE is what is causing the difference.

[billhoffman]

OK, I have another possible patch. This one does not break the build.

diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index 9878a9dad78d..b3ef6aaf5d06 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -6205,6 +6205,11 @@ std::string Driver::GetFilePath(StringRef Name, const Too
lChain &TC) const {
   if (llvm::sys::fs::exists(Twine(R)))
     return std::string(R);
 
+  SmallString<128> R2(ResourceDir);
+  llvm::sys::path::append(R2, "..", "..", Name);
+  if (llvm::sys::fs::exists(Twine(R2)))
+    return std::string(R2);
+
   SmallString<128> P(TC.getCompilerRTPath());
   llvm::sys::path::append(P, Name);
   if (llvm::sys::fs::exists(Twine(P)))

[ChuanqiXu9]

OK, I have another possible patch. This one does not break the build.

diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index 9878a9dad78d..b3ef6aaf5d06 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -6205,6 +6205,11 @@ std::string Driver::GetFilePath(StringRef Name, const Too
lChain &TC) const {
   if (llvm::sys::fs::exists(Twine(R)))
     return std::string(R);
 
+  SmallString<128> R2(ResourceDir);
+  llvm::sys::path::append(R2, "..", "..", Name);
+  if (llvm::sys::fs::exists(Twine(R2)))
+    return std::string(R2);
+
   SmallString<128> P(TC.getCompilerRTPath());
   llvm::sys::path::append(P, Name);
   if (llvm::sys::fs::exists(Twine(P)))

Although it might be a little bit add hoc, it is much better than the existing status. I think you can send a PR after you add a test. For the test, you can take a look at: clang/test/Driver/modules-print-library-module-manifest-path.cpp

[billhoffman]

Yeah, it did feel a bit ugly. But I think this is a new place for files like this to be found. There is actually some code right above this that looks in "..", just not "../..". Clearly the test is bad, I will see if I can get a test in there that fails the way it is now.