Clang does not resolve path to linker executable using PATH env variable

[artem]

For example, if you create a symbolic link from /usr/bin/lld-18 to /usr/local/bin/ld, GCC successfully hooks it up. However, Clang does not, and running clang -v reveals that it unconditionally calls /usr/bin/ld.

I'm not sure whether such behavior is intended, but at least it is very confusing and frustrating to discover.

[artem]

user@hostname:~$ which ld
/home/user/bin/ld
user@hostname:~$ clang++-18 -v test.cpp -o test
Ubuntu clang version 18.1.0 (++20240128073201+27654471cc7a-1~exp1~20240128073233.16)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib/llvm-18/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -dumpdir test- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/user -v -fcoverage-compilation-dir=/home/user -resource-dir /usr/lib/llvm-18/lib/clang/18 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/x86_64-linux-gnu/c++/13 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/backward -internal-isystem /usr/lib/llvm-18/lib/clang/18/include -internal-isystem /usr/local/include -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/test-2932cb.o -x c++ test.cpp
clang -cc1 version 18.1.0 based upon LLVM 18.1.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13
 /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/x86_64-linux-gnu/c++/13
 /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/backward
 /usr/lib/llvm-18/lib/clang/18/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o test /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/13/crtbeginS.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/lib -L/usr/lib /tmp/test-2932cb.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/x86_64-linux-gnu/13/crtendS.o /lib/x86_64-linux-gnu/crtn.o
user@hostname:~$

[llvmbot]

@llvm/issue-subscribers-clang-driver

Author: Artem Labazov (artem)

For example, if you create a symbolic link from /usr/bin/lld-18 to /usr/local/bin/ld, GCC successfully hooks it up. However, Clang does not, and running `clang -v` reveals that it unconditionally calls `/usr/bin/ld`.

I'm not sure whether such behavior is intended, but at least it is very confusing and frustrating to discover.

[tru]

@MaskRay has done quite a bit of work around this area.

I am weakly against looking up the linker from the PATH, I don't think it fixes any real-world problems. But it might be worth it for the compatibility with GCC.

[artem]

Quoting LLD website:

The easiest way to do that is to overwrite the default linker. After installing LLD to somewhere on your disk, you can create a symbolic link by doing ln -s /path/to/ld.lld /usr/bin/ld so that /usr/bin/ld is resolved to LLD.

However, such thing is infeasible in environments without superuser access. Instead, doing ln -s /usr/bin/ld.lld ~/bin/ld is both convenient and elegant.

[MaskRay]

Clang does consult PATH, but its priority is low. It will usually use ld beside the clang executable. You can place a lld symlink in a directory and specify -B.

ln -s path/to/lld bin/ld
clang -### -Bbin a.o

% PATH=/tmp/c/bin:$PATH fclang a.o -fuse-ld=lld '-###'
clang version 19.0.0git
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /tmp/Rel/bin
 "/tmp/c/bin/ld.lld" ...

[artem]

You can place a lld symlink in a directory and specify -B.

Unfortunately, it does not make LLD the "default system linker", every project here and there has its own way to pass extra flags to compiler (or none at all without patching its build recipes).

Clang does consult PATH, but its priority is low. It will usually use ld beside the clang executable.

Thanks! I have confirmed that creating an extra clang symlink kinda works, but is there any strong reason behind such behavior? This workaround requires an extra counterintuitive step and is misaligned with what GCC driver does.

[MaskRay]

I think this is a Ubuntu packaging problem. Debian's is good:)

Clang basically calls GetProgramPath("ld.lld") if -fuse-ld=lld is requested and otherwise GetProgramPath("ld").

string GetProgramPath(name) {
  for (b : OPT_B) {
    cand = is_directory(b) ? join(b, name) : b + name;
    if (is_executable(cand))
      return cand;
  }
  for (TargetSpecificExecutable : TargetSpecificExecutables) { // e.g. aarch64-linux-gnu-$name, $name
    for (auto dir : TC.getProgramPaths()) {  // -ccc-install-dir, /usr/lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/bin
      cand = join(b, name);
      if (is_executable(cand))
        return cand;
    }
    for (auto dir : split(getenv("PATH"))) {
      cand = join(b, name);
      if (is_executable(cand))
        return cand;
    }
  }
  return name;
}

% mkdir -p bin
% ln -sr ~/Stable/bin/lld bin/ld   # ensure it is executable
% PATH=$PWD/bin:$PATH /usr/lib/llvm-16/bin/clang a.o '-###'
Debian clang version 16.0.6 (16)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-16/bin
 "/tmp/c/bin/ld" ...

Note that InstalledDir is /usr/lib/llvm-16/bin, so Clang will try finding ld there (not found), and fall back to PATH.

In your case, InstalledDir: /usr/bin, Clang will prefer /usr/bin/ld.

InstalledDir is preferred over PATH because this gives vendors a way to ensure executables shipped with the clang executable have a higher precedence than PATH.

[artem]

I think this is a Ubuntu packaging problem. Debian's is good:)

Debian is affected as well: there're two symlinks: /usr/bin/clang -> ../lib/llvm-XY/bin/clang and /usr/bin/clang-XY -> ../lib/llvm-XY/bin/clang:

user@host:~$ clang-17 -v
Debian clang version 17.0.6 (++20231128093145+6009708b4367-1~exp1~20231128093253.72)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

If you do not add /usr/lib/llvm-XX/bin to PATH (default) and set CC to either clang or clang-18, then the system finds the driver in /usr/bin, Clang considers it the InstalledDir and ld from PATH gets ignored.

As you suggested, adding /usr/lib/llvm-XX/bin to the PATH no longer makes Сlang toolchain think that it "ships" a linker. But this way adding a directory without ld to PATH leads to ld being resolved to a totally different place, which does not sound robust :/

Put another way, it's weird that clang considers cmdline basedir a part of toolchain, no matter where the actual binary is located on the filesystem. From user's perspective, Calling /usr/bin/clang-16 should have identical to calling /usr/lib/llvm-16/bin/clang, since they are the same. Intuitively, InstalledDir should never be /usr/bin but /usr/lib/llvm-XY/bin, the former contains many other binaries from other packages.

InstalledDir is preferred over PATH because this gives vendors a way to ensure executables shipped with the clang executable have a higher precedence than PATH.

GCC does not exhibit this problem since there are COMPILER_PATH and PATH, the former has higher precedence and does not include /usr/bin/:
COMPILER_PATH=/usr/libexec/gcc/x86_64-linux-gnu/13/:/usr/libexec/gcc/x86_64-linux-gnu/13/:/usr/libexec/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/13/:/usr/lib/gcc/x86_64-linux-gnu/

After all, if the current situation is not to change, it would be great to at least document it (please forgive me if it already is) to warn users about this pitfall in the future.

[MaskRay]

Debian is affected as well: there're two symlinks: /usr/bin/clang -> ../lib/llvm-XY/bin/clang and /usr/bin/clang-XY -> ../lib/llvm-XY/bin/clang:

Agree. Debian has the problem as well. InstalledDir: /usr/bin ideally should be /usr/lib/llvm-XY/bin.
I think this is related to InstalledDir/Dir inconsistency in the default -canonical-prefixes mode.

I've created #80527 , which should update InstalledDir to /usr/lib/llvm-XY/bin.
Since /usr/lib/llvm-XY/bin does not contain ld, setting PATH to a directory containing ld will override Clang-invoked ld.

After all, if the current situation is not to change, it would be great to at least document it (please forgive me if it already is) to warn users about this pitfall in the future.

I agree that some documentation will be nice. Perhaps clang/docs/UsersManual.rst can have a paragraph, but the topic would be quite subtle. Each ToolChain in clang/lib/Driver/ToolChains/ may be slight different:(