offload: Fails to build libomp with `missing and no known rule to make it`

[sylvestre]

with

cmake -S ../llvm/ -B build-llvm \
-G Ninja \
-DCMAKE_INSTALL_PREFIX=/usr/lib/llvm-19 \
-DLLVM_VERSION_SUFFIX= \
-DCMAKE_SUPPRESS_REGENERATION=ON \
-DCMAKE_BUILD_TYPE=Release \
-DPACKAGE_VENDOR=Debian \
-DENABLE_LINKER_BUILD_ID=ON \
-DLLVM_TARGETS_TO_BUILD=Native \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;cross-project-tests;mlir;bolt;flang;polly;openmp;lldb" \
-DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libc;libunwind;offload" \
-DLLVM_ENABLE_PIC=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_BUILD_DOCS=OFF \
-DLLVM_INCLUDE_GO_TESTS=OFF \
-DLLVM_USE_RELATIVE_PATHS_IN_FILES=ON \
-DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \
-DLLVM_ENABLE_CURL=ON \
-DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=ON \
-DCLANG_PLUGIN_SUPPORT=OFF \
-DCLANG_BUILD_EXAMPLES=OFF \
-DCLANG_DEFAULT_LINKER=ld \
-DCLANG_DEFAULT_RTLIB=libgcc \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_INCLUDE_TESTS=OFF \
-DCOMPILER_RT_USE_LIBCXX=OFF \
-DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \
-DLIBUNWIND_INSTALL_LIBRARY=OFF \
-DOPENMP_ENABLE_LIBOMP_PROFILING=OFF \
-DOPENMP_ENABLE_LIBOMPTARGET_PROFILING=OFF \
-DLIBOMP_ENABLE_RTTI=OFF \
-DLIBOMP_OMPT_SUPPORT=OFF \
-DCLANG_FORCE_MATCHING_LIBCLANG_SOVERSION=ON \
-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \
-DLLVM_LIBC_GPU_BUILD=ON \
 -DLLVM_USE_LINKER=gold -DLLVM_BINUTILS_INCDIR=/usr/include/ -DLIBCXXABI_USE_LLVM_UNWINDER=ON -DLIBOMP_LIBFLAGS="-lm" -DOPENMP_USE_LLVM_UNWINDER=ON  \
-DCLANG_ENABLE_BOOTSTRAP=ON

and

cmake --build build-llvm -j 13 --target stage2

The second stage fails with:

-- Configuring done (33.6s)
-- Generating done (1.3s)
-- Build files have been written to: /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins
[3159/3170] cd /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins && /usr/bin/cmake --build .
Change Dir: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins'

Run Build Command(s): /usr/bin/ninja -v
ninja: error: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomp.so', needed by '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomptarget.so.19.0', missing and no known rule to make it

FAILED: runtimes/runtimes-stamps/runtimes-build /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-stamps/runtimes-build

@jdoerfert @jhuber6 rings a bell ?

[llvmbot]

@llvm/issue-subscribers-openmp

Author: Sylvestre Ledru (sylvestre)

with ``` cmake -S ../llvm/ -B build-llvm \ -G Ninja \ -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm-19 \ -DLLVM_VERSION_SUFFIX= \ -DCMAKE_SUPPRESS_REGENERATION=ON \ -DCMAKE_BUILD_TYPE=Release \ -DPACKAGE_VENDOR=Debian \ -DENABLE_LINKER_BUILD_ID=ON \ -DLLVM_TARGETS_TO_BUILD=Native \ -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;cross-project-tests;mlir;bolt;flang;polly;openmp;lldb" \ -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libc;libunwind;offload" \ -DLLVM_ENABLE_PIC=ON \ -DLLVM_ENABLE_RTTI=ON \ -DLLVM_BUILD_DOCS=OFF \ -DLLVM_INCLUDE_GO_TESTS=OFF \ -DLLVM_USE_RELATIVE_PATHS_IN_FILES=ON \ -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ -DLLVM_ENABLE_CURL=ON \ -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=ON \ -DCLANG_PLUGIN_SUPPORT=OFF \ -DCLANG_BUILD_EXAMPLES=OFF \ -DCLANG_DEFAULT_LINKER=ld \ -DCLANG_DEFAULT_RTLIB=libgcc \ -DCOMPILER_RT_BUILD_XRAY=OFF \ -DCOMPILER_RT_INCLUDE_TESTS=OFF \ -DCOMPILER_RT_USE_LIBCXX=OFF \ -DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \ -DLIBUNWIND_INSTALL_LIBRARY=OFF \ -DOPENMP_ENABLE_LIBOMP_PROFILING=OFF \ -DOPENMP_ENABLE_LIBOMPTARGET_PROFILING=OFF \ -DLIBOMP_ENABLE_RTTI=OFF \ -DLIBOMP_OMPT_SUPPORT=OFF \ -DCLANG_FORCE_MATCHING_LIBCLANG_SOVERSION=ON \ -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \ -DLLVM_LIBC_GPU_BUILD=ON \ -DCMAKE_C_COMPILER_LAUNCHER=/opt/sccache//sccache -DCMAKE_CXX_COMPILER_LAUNCHER=/opt/sccache//sccache -DLLVM_USE_LINKER=gold -DLLVM_BINUTILS_INCDIR=/usr/include/ -DLIBCXXABI_USE_LLVM_UNWINDER=ON -DLIBOMP_LIBFLAGS="-lm" -DOPENMP_USE_LLVM_UNWINDER=ON \ -DCLANG_ENABLE_BOOTSTRAP=ON ```

and

cmake --build build-llvm -j 13 --target stage2

The second stage fails with:

-- Configuring done (33.6s)
-- Generating done (1.3s)
-- Build files have been written to: /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins
[3159/3170] cd /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins && /usr/bin/cmake --build .
Change Dir: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins'

Run Build Command(s): /usr/bin/ninja -v
ninja: error: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomp.so', needed by '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomptarget.so.19.0', missing and no known rule to make it

FAILED: runtimes/runtimes-stamps/runtimes-build /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-stamps/runtimes-build

@jdoerfert @jhuber6 rings a bell ?

[llvmbot]

@llvm/issue-subscribers-offload

Author: Sylvestre Ledru (sylvestre)

with ``` cmake -S ../llvm/ -B build-llvm \ -G Ninja \ -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm-19 \ -DLLVM_VERSION_SUFFIX= \ -DCMAKE_SUPPRESS_REGENERATION=ON \ -DCMAKE_BUILD_TYPE=Release \ -DPACKAGE_VENDOR=Debian \ -DENABLE_LINKER_BUILD_ID=ON \ -DLLVM_TARGETS_TO_BUILD=Native \ -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;cross-project-tests;mlir;bolt;flang;polly;openmp;lldb" \ -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libc;libunwind;offload" \ -DLLVM_ENABLE_PIC=ON \ -DLLVM_ENABLE_RTTI=ON \ -DLLVM_BUILD_DOCS=OFF \ -DLLVM_INCLUDE_GO_TESTS=OFF \ -DLLVM_USE_RELATIVE_PATHS_IN_FILES=ON \ -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ -DLLVM_ENABLE_CURL=ON \ -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=ON \ -DCLANG_PLUGIN_SUPPORT=OFF \ -DCLANG_BUILD_EXAMPLES=OFF \ -DCLANG_DEFAULT_LINKER=ld \ -DCLANG_DEFAULT_RTLIB=libgcc \ -DCOMPILER_RT_BUILD_XRAY=OFF \ -DCOMPILER_RT_INCLUDE_TESTS=OFF \ -DCOMPILER_RT_USE_LIBCXX=OFF \ -DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \ -DLIBUNWIND_INSTALL_LIBRARY=OFF \ -DOPENMP_ENABLE_LIBOMP_PROFILING=OFF \ -DOPENMP_ENABLE_LIBOMPTARGET_PROFILING=OFF \ -DLIBOMP_ENABLE_RTTI=OFF \ -DLIBOMP_OMPT_SUPPORT=OFF \ -DCLANG_FORCE_MATCHING_LIBCLANG_SOVERSION=ON \ -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \ -DLLVM_LIBC_GPU_BUILD=ON \ -DCMAKE_C_COMPILER_LAUNCHER=/opt/sccache//sccache -DCMAKE_CXX_COMPILER_LAUNCHER=/opt/sccache//sccache -DLLVM_USE_LINKER=gold -DLLVM_BINUTILS_INCDIR=/usr/include/ -DLIBCXXABI_USE_LLVM_UNWINDER=ON -DLIBOMP_LIBFLAGS="-lm" -DOPENMP_USE_LLVM_UNWINDER=ON \ -DCLANG_ENABLE_BOOTSTRAP=ON ```

and

cmake --build build-llvm -j 13 --target stage2

The second stage fails with:

-- Configuring done (33.6s)
-- Generating done (1.3s)
-- Build files have been written to: /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins
[3159/3170] cd /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins && /usr/bin/cmake --build .
Change Dir: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins'

Run Build Command(s): /usr/bin/ninja -v
ninja: error: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomp.so', needed by '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomptarget.so.19.0', missing and no known rule to make it

FAILED: runtimes/runtimes-stamps/runtimes-build /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-stamps/runtimes-build

@jdoerfert @jhuber6 rings a bell ?

[jhuber6]

I think it's expected to put openmp in runtimes instead. The old code all expected that openmp was in the same CMake invocation, but now that they're split it's not going to have access to the same targets.

[sylvestre]

ok, maybe we should detect this case and break cmake then :)

[jhuber6]

I think we should be able to make this work in any case, since openmp should be a valid project now that it's just libomp.

[shiltian]

I think we should be able to make this work in any case, since openmp should be a valid project now that it's just libomp.

Yeah, we still need to make sure libomp works as before, no matter how libomptarget changes.

[estewart08]

One problem is with https://github.com/llvm/llvm-project/blob/main/llvm/runtimes/CMakeLists.txt#L561. This adds the runtimes to clang-bootstrap-deps. If openmp is on the LLVM_ENABLE_PROJECTS list instead of LLVM_ENABLE_RUNTIMES, then the configuration of the runtimes will occur before the openmp project. See #90793 for possible fix.

[jhuber6]

Is this still an issue since making offload a no-op on unsupported architectures?

[jhuber6]

Closing because it should be fixed and I haven't heard anything back.