Bug in getInstalledDir() prevents picking up the correct headers when clang is started via a link

[ilg-ul]

In clang/tools/driver/driver.cpp, the function SetInstallDir() tries to determine the location where the toolchain is installed, basically by taking the parent folder of the executable absolute path:

llvm-project/clang/tools/driver/driver.cpp

Line 344 in 9bbec0a

StringRef InstalledPathParent(llvm::sys::path::parent_path(InstalledPath));

This works for regular use cases, but fails if the executable is started via a soft link, when this code returns the parent of the link, not the parent of the executable after following the link(s).

This case is specific to the npm/xpm ecosystem, where multi-version dependencies are managed by creating soft links into a local .bin folder instead of adding lots of folders to the PATH.

Below is a functional example of a simple compile step on macOS, when the compiler is invoked with the absolute path:

% /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang++ -v hello.cpp -stdlib=libc++
xPack x86_64 clang version 15.0.7 (https://github.com/xpack-dev-tools/clang-xpack 9b1ff65945b1aaddfe7c0c4d99001ebca5d67b03)
Target: x86_64-apple-darwin21.6.0
Thread model: posix

InstalledDir: /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin <-- Correct!

 "/Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang-15" -cc1 -triple x86_64-apple-macosx12.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=all -ffp-contract=on -fno-rounding-math -funwind-tables=2 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 409.12 -v -fcoverage-compilation-dir=/Users/ilg/tmp -resource-dir /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -stdlib=libc++ -internal-isystem /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin/../include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -fdeprecated-macro -fdebug-compilation-dir=/Users/ilg/tmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-1fa361.o -x c++ hello.cpp
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-apple-darwin21.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:

 /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin/../include/c++/v1 <-- Correct

 /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
 "/usr/bin/ld" -demangle -lto_library /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 12.0.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-1fa361.o -lc++ -lSystem /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/lib/darwin/libclang_rt.osx.a

As it can be seen, the correct headers in .content/bin/../include/c++/v1 were included.

When the compiler is invoked via a link, it computes an invalid InstalledDir, then complains about the missing C++ headers folder and finally silently picks the wrong system headers:

% ln -s /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang++ ~/tmp/clang++
%  ~/tmp/clang++ -v hello.cpp -stdlib=libc++
xPack x86_64 clang version 15.0.7 (https://github.com/xpack-dev-tools/clang-xpack 9b1ff65945b1aaddfe7c0c4d99001ebca5d67b03)
Target: x86_64-apple-darwin21.6.0
Thread model: posix

InstalledDir: /Users/ilg/tmp/. <--- Incorrect!
ignoring nonexistent directory "/Users/ilg/tmp/./../include/c++/v1" <--- Bad!

 "/Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang-15" -cc1 -triple x86_64-apple-macosx12.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=all -ffp-contract=on -fno-rounding-math -funwind-tables=2 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 409.12 -v -fcoverage-compilation-dir=/Users/ilg/tmp -resource-dir /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -fdeprecated-macro -fdebug-compilation-dir=/Users/ilg/tmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-a87934.o -x c++ hello.cpp
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-apple-darwin21.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:

 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 <--- Wrong!

 /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
 "/usr/bin/ld" -demangle -lto_library /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 12.0.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-a87934.o -lc++ -lSystem /Users/ilg/Library/xPacks/@xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/lib/darwin/libclang_rt.osx.a

Using the system headers instead of the toolchain headers may have very subtle consequences, sometimes leading to compile errors which are hard to diagnose.

The correct implementation of SetInstallDir() is platform specific, and requires invoking some system calls, like proc_pidpath() on macOS.

In the xPack LLVM clang binary distribution I patched driver.cpp and fixed the problem.

Attached is the small patch.

If you are interrested, I can try to submit it as a pull request.

[ilg-ul]

The patch apparently applied correctly on 17.0.1 too.

[ilg-ul]

Could any of the maintainers advice if a pull request to fix this would be accepted?

[llvmbot]

@llvm/issue-subscribers-clang-driver

In `clang/tools/driver/driver.cpp`, the function `SetInstallDir()` tries to determine the location where the toolchain is installed, basically by taking the parent folder of the executable absolute path:

llvm-project/clang/tools/driver/driver.cpp

Line 344 in 9bbec0a

StringRef InstalledPathParent(llvm::sys::path::parent_path(InstalledPath));

This works for regular use cases, but fails if the executable is started via a soft link, when this code returns the parent of the link, not the parent of the executable after following the link(s).

This case is specific to the npm/xpm ecosystem, where multi-version dependencies are managed by creating soft links into a local .bin folder instead of adding lots of folders to the PATH.

Below is a functional example of a simple compile step on macOS, when the compiler is invoked with the absolute path:

% /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang++ -v hello.cpp -stdlib=libc++
xPack x86_64 clang version 15.0.7 (https://github.com/xpack-dev-tools/clang-xpack 9b1ff65945b1aaddfe7c0c4d99001ebca5d67b03)
Target: x86_64-apple-darwin21.6.0
Thread model: posix

InstalledDir: /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin &lt;-- Correct!

 "/Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang-15" -cc1 -triple x86_64-apple-macosx12.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=all -ffp-contract=on -fno-rounding-math -funwind-tables=2 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 409.12 -v -fcoverage-compilation-dir=/Users/ilg/tmp -resource-dir /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -stdlib=libc++ -internal-isystem /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin/../include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -fdeprecated-macro -fdebug-compilation-dir=/Users/ilg/tmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-1fa361.o -x c++ hello.cpp
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-apple-darwin21.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include &lt;...&gt; search starts here:

 /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin/../include/c++/v1 &lt;-- Correct

 /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
 "/usr/bin/ld" -demangle -lto_library /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 12.0.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-1fa361.o -lc++ -lSystem /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/lib/darwin/libclang_rt.osx.a

As it can be seen, the correct headers in .content/bin/../include/c++/v1 were included.

When the compiler is invoked via a link, it computes an invalid InstalledDir, then complains about the missing C++ headers folder and finally silently picks the wrong system headers:

% ln -s /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang++ ~/tmp/clang++
%  ~/tmp/clang++ -v hello.cpp -stdlib=libc++
xPack x86_64 clang version 15.0.7 (https://github.com/xpack-dev-tools/clang-xpack 9b1ff65945b1aaddfe7c0c4d99001ebca5d67b03)
Target: x86_64-apple-darwin21.6.0
Thread model: posix

InstalledDir: /Users/ilg/tmp/. &lt;--- Incorrect!
ignoring nonexistent directory "/Users/ilg/tmp/./../include/c++/v1" &lt;--- Bad!

 "/Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/bin/clang-15" -cc1 -triple x86_64-apple-macosx12.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=all -ffp-contract=on -fno-rounding-math -funwind-tables=2 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 409.12 -v -fcoverage-compilation-dir=/Users/ilg/tmp -resource-dir /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -fdeprecated-macro -fdebug-compilation-dir=/Users/ilg/tmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-a87934.o -x c++ hello.cpp
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-apple-darwin21.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include &lt;...&gt; search starts here:

 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 &lt;--- Wrong!

 /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
 "/usr/bin/ld" -demangle -lto_library /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 12.0.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out /var/folders/gr/13tt3vcd7m1gnbhwtkmf5cnw0000gn/T/hello-a87934.o -lc++ -lSystem /Users/ilg/Library/xPacks/@<!-- -->xpack-dev-tools/clang/15.0.7-3.1/.content/lib/clang/15.0.7/lib/darwin/libclang_rt.osx.a

Using the system headers instead of the toolchain headers may have very subtle consequences, sometimes leading to compile errors which are hard to diagnose.

The correct implementation of SetInstallDir() is platform specific, and requires invoking some system calls, like proc_pidpath() on macOS.

In the xPack LLVM clang binary distribution I patched driver.cpp and fixed the problem.

Attached is the small patch.

If you are interrested, I can try to submit it as a pull request.

[Endilll]

@ilg-ul Go ahead and submit a pull request! If anything, this should help to start a discussion.

[ilg-ul]

@Endilll, I submitted a pull request, and there were some failed tests, but I'm not sure they are related to my patch.

Could you advice how to proceed?

[Endilll]

I responded in the PR.

[ilg-ul]

Thank you, I'll try to replicate the failed tests locally.

[ilg-ul]

Fixed by: a32a2b2