[libc] Rework the GPU build to be a regular target #81921
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jhuber6
requested review from
arsenm,
Artem-B,
gchatelet,
jdoerfert,
JonChesterfield,
jplehr,
lntue,
michaelrj-google,
nickdesaulniers,
petrhosek,
ronlieb and
shiltian
llvmbot
added
cmake
|
@llvm/pr-subscribers-clang-driver @llvm/pr-subscribers-backend-amdgpu Author: Joseph Huber (jhuber6) ChangesSummary: This patch completely reworks how the GPU build is built and targeted. The new appraoch is to handle the GPU builds as strict cross-compiling The new expected way to build the GPU libc is with The second step was reworking how we generated the embedded GPU library We unfortunately still require that the NVPTX target has an This approach is vastly suprerior in every way, allowing us to treat the Depends on #81557 Patch is 93.74 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/81921.diff 50 Files Affected:
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index 0fd7b8424eb4ba..09f5c21528b141 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -875,7 +875,7 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
// LowerMatrixIntrinsicsPass, which is transitively called by
// buildThinLTODefaultPipeline under EnableMatrix.
if ((IsThinLTO || IsFatLTO || IsUnifiedLTO) &&
- Args.hasArg(options::OPT_fenable_matrix))
+ Args.hasArg(options::OPT_fenable_matrix))
CmdArgs.push_back(
Args.MakeArgString(Twine(PluginOptPrefix) + "-enable-matrix"));
@@ -1087,10 +1087,41 @@ static void addOpenMPDeviceLibC(const ToolChain &TC, const ArgList &Args,
"llvm-libc-decls");
bool HasLibC = llvm::sys::fs::exists(LibCDecls) &&
llvm::sys::fs::is_directory(LibCDecls);
- if (Args.hasFlag(options::OPT_gpulibc, options::OPT_nogpulibc, HasLibC)) {
- CmdArgs.push_back("-lcgpu");
- CmdArgs.push_back("-lmgpu");
+ if (!Args.hasFlag(options::OPT_gpulibc, options::OPT_nogpulibc, HasLibC))
+ return;
+
+ // We don't have access to the offloading toolchains here, so determine from
+ // the arguments if we have any active NVPTX or AMDGPU toolchains.
+ llvm::DenseSet<const char *> Libraries;
+ if (const Arg *Targets = Args.getLastArg(options::OPT_fopenmp_targets_EQ)) {
+ if (llvm::any_of(Targets->getValues(),
+ [](auto S) { return llvm::Triple(S).isAMDGPU(); })) {
+ Libraries.insert("-lcgpu-amdgpu");
+ Libraries.insert("-lmgpu-amdgpu");
+ }
+ if (llvm::any_of(Targets->getValues(),
+ [](auto S) { return llvm::Triple(S).isNVPTX(); })) {
+ Libraries.insert("-lcgpu-nvptx");
+ Libraries.insert("-lmgpu-nvptx");
+ }
+ }
+
+ for (StringRef Arch : Args.getAllArgValues(options::OPT_offload_arch_EQ)) {
+ if (llvm::any_of(llvm::split(Arch, ","), [](StringRef Str) {
+ return IsAMDGpuArch(StringToCudaArch(Str));
+ })) {
+ Libraries.insert("-lcgpu-amdgpu");
+ Libraries.insert("-lmgpu-amdgpu");
+ }
+ if (llvm::any_of(llvm::split(Arch, ","), [](StringRef Str) {
+ return IsNVIDIAGpuArch(StringToCudaArch(Str));
+ })) {
+ Libraries.insert("-lcgpu-nvptx");
+ Libraries.insert("-lmgpu-nvptx");
+ }
}
+
+ llvm::append_range(CmdArgs, Libraries);
}
void tools::addOpenMPRuntimeLibraryPath(const ToolChain &TC,
@@ -1152,7 +1183,7 @@ bool tools::addOpenMPRuntime(ArgStringList &CmdArgs, const ToolChain &TC,
CmdArgs.push_back("-Bdynamic");
if (RTKind == Driver::OMPRT_GOMP && GompNeedsRT)
- CmdArgs.push_back("-lrt");
+ CmdArgs.push_back("-lrt");
if (IsOffloadingHost)
CmdArgs.push_back("-lomptarget");
@@ -1311,10 +1342,12 @@ static void addSanitizerRuntime(const ToolChain &TC, const ArgList &Args,
bool IsShared, bool IsWhole) {
// Wrap any static runtimes that must be forced into executable in
// whole-archive.
- if (IsWhole) CmdArgs.push_back("--whole-archive");
+ if (IsWhole)
+ CmdArgs.push_back("--whole-archive");
CmdArgs.push_back(TC.getCompilerRTArgString(
Args, Sanitizer, IsShared ? ToolChain::FT_Shared : ToolChain::FT_Static));
- if (IsWhole) CmdArgs.push_back("--no-whole-archive");
+ if (IsWhole)
+ CmdArgs.push_back("--no-whole-archive");
if (IsShared) {
addArchSpecificRPath(TC, Args, CmdArgs);
@@ -1381,8 +1414,7 @@ void tools::linkSanitizerRuntimeDeps(const ToolChain &TC,
TC.getTriple().getOS() != llvm::Triple::RTEMS)
CmdArgs.push_back("-ldl");
// Required for backtrace on some OSes
- if (TC.getTriple().isOSFreeBSD() ||
- TC.getTriple().isOSNetBSD() ||
+ if (TC.getTriple().isOSFreeBSD() || TC.getTriple().isOSNetBSD() ||
TC.getTriple().isOSOpenBSD())
CmdArgs.push_back("-lexecinfo");
// There is no libresolv on Android, FreeBSD, OpenBSD, etc. On musl
@@ -1594,7 +1626,8 @@ bool tools::addSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
return !StaticRuntimes.empty() || !NonWholeStaticRuntimes.empty();
}
-bool tools::addXRayRuntime(const ToolChain&TC, const ArgList &Args, ArgStringList &CmdArgs) {
+bool tools::addXRayRuntime(const ToolChain &TC, const ArgList &Args,
+ ArgStringList &CmdArgs) {
if (Args.hasArg(options::OPT_shared))
return false;
@@ -1619,8 +1652,7 @@ void tools::linkXRayRuntimeDeps(const ToolChain &TC,
CmdArgs.push_back("-lrt");
CmdArgs.push_back("-lm");
- if (!TC.getTriple().isOSFreeBSD() &&
- !TC.getTriple().isOSNetBSD() &&
+ if (!TC.getTriple().isOSFreeBSD() && !TC.getTriple().isOSNetBSD() &&
!TC.getTriple().isOSOpenBSD())
CmdArgs.push_back("-ldl");
}
@@ -1911,19 +1943,19 @@ tools::ParsePICArgs(const ToolChain &ToolChain, const ArgList &Args) {
bool EmbeddedPISupported;
switch (Triple.getArch()) {
- case llvm::Triple::arm:
- case llvm::Triple::armeb:
- case llvm::Triple::thumb:
- case llvm::Triple::thumbeb:
- EmbeddedPISupported = true;
- break;
- default:
- EmbeddedPISupported = false;
- break;
+ case llvm::Triple::arm:
+ case llvm::Triple::armeb:
+ case llvm::Triple::thumb:
+ case llvm::Triple::thumbeb:
+ EmbeddedPISupported = true;
+ break;
+ default:
+ EmbeddedPISupported = false;
+ break;
}
bool ROPI = false, RWPI = false;
- Arg* LastROPIArg = Args.getLastArg(options::OPT_fropi, options::OPT_fno_ropi);
+ Arg *LastROPIArg = Args.getLastArg(options::OPT_fropi, options::OPT_fno_ropi);
if (LastROPIArg && LastROPIArg->getOption().matches(options::OPT_fropi)) {
if (!EmbeddedPISupported)
ToolChain.getDriver().Diag(diag::err_drv_unsupported_opt_for_target)
@@ -1952,7 +1984,7 @@ tools::ParsePICArgs(const ToolChain &ToolChain, const ArgList &Args) {
if (ABIName == "n64")
PIC = true;
// When targettng MIPS with -mno-abicalls, it's always static.
- if(Args.hasArg(options::OPT_mno_abicalls))
+ if (Args.hasArg(options::OPT_mno_abicalls))
return std::make_tuple(llvm::Reloc::Static, 0U, false);
// Unlike other architectures, MIPS, even with -fPIC/-mxgot/multigot,
// does not use PIC level 2 for historical reasons.
@@ -2114,7 +2146,8 @@ enum class LibGccType { UnspecifiedLibGcc, StaticLibGcc, SharedLibGcc };
static LibGccType getLibGccType(const ToolChain &TC, const Driver &D,
const ArgList &Args) {
if (Args.hasArg(options::OPT_static_libgcc) ||
- Args.hasArg(options::OPT_static) || Args.hasArg(options::OPT_static_pie) ||
+ Args.hasArg(options::OPT_static) ||
+ Args.hasArg(options::OPT_static_pie) ||
// The Android NDK only provides libunwind.a, not libunwind.so.
TC.getTriple().isAndroid())
return LibGccType::StaticLibGcc;
@@ -2482,11 +2515,10 @@ static void GetSDLFromOffloadArchive(
return;
StringRef Prefix = isBitCodeSDL ? "libbc-" : "lib";
- std::string OutputLib =
- D.GetTemporaryPath(Twine(Prefix + llvm::sys::path::filename(Lib) + "-" +
- Arch + "-" + Target)
- .str(),
- "a");
+ std::string OutputLib = D.GetTemporaryPath(
+ Twine(Prefix + llvm::sys::path::filename(Lib) + "-" + Arch + "-" + Target)
+ .str(),
+ "a");
C.addTempFile(C.getArgs().MakeArgString(OutputLib));
@@ -2699,8 +2731,8 @@ void tools::addMachineOutlinerArgs(const Driver &D,
}
};
- if (Arg *A = Args.getLastArg(options::OPT_moutline,
- options::OPT_mno_outline)) {
+ if (Arg *A =
+ Args.getLastArg(options::OPT_moutline, options::OPT_mno_outline)) {
if (A->getOption().matches(options::OPT_moutline)) {
// We only support -moutline in AArch64 and ARM targets right now. If
// we're not compiling for these, emit a warning and ignore the flag.
diff --git a/clang/test/Driver/openmp-offload-gpu.c b/clang/test/Driver/openmp-offload-gpu.c
index bccc5fd9483ac8..3620bf5340fe0e 100644
--- a/clang/test/Driver/openmp-offload-gpu.c
+++ b/clang/test/Driver/openmp-offload-gpu.c
@@ -394,13 +394,21 @@
// RUN: %clang -### --target=x86_64-unknown-linux-gnu -fopenmp=libomp \
// RUN: --libomptarget-nvptx-bc-path=%S/Inputs/libomptarget/libomptarget-nvptx-test.bc \
// RUN: --cuda-path=%S/Inputs/CUDA_102/usr/local/cuda \
-// RUN: --offload-arch=sm_52 -gpulibc -nogpuinc %s 2>&1 \
+// RUN: --offload-arch=sm_52,gfx90a -gpulibc -nogpuinc %s 2>&1 \
// RUN: | FileCheck --check-prefix=LIBC-GPU %s
-// LIBC-GPU: "-lcgpu"{{.*}}"-lmgpu"
+// RUN: %clang -### --target=x86_64-unknown-linux-gnu -fopenmp=libomp \
+// RUN: --libomptarget-nvptx-bc-path=%S/Inputs/libomptarget/libomptarget-nvptx-test.bc \
+// RUN: --cuda-path=%S/Inputs/CUDA_102/usr/local/cuda \
+// RUN: -fopenmp-targets=nvptx64-nvidia-cuda,amdgcn-amd-amdhsa -gpulibc -nogpuinc %s 2>&1 \
+// RUN: | FileCheck --check-prefix=LIBC-GPU %s
+// LIBC-GPU-DAG: "-lcgpu-amdgpu"
+// LIBC-GPU-DAG: "-lmgpu-amdgpu"
+// LIBC-GPU-DAG: "-lcgpu-nvptx"
+// LIBC-GPU-DAG: "-lmgpu-nvptx"
// RUN: %clang -### --target=x86_64-unknown-linux-gnu -fopenmp=libomp \
// RUN: --libomptarget-nvptx-bc-path=%S/Inputs/libomptarget/libomptarget-nvptx-test.bc \
// RUN: --cuda-path=%S/Inputs/CUDA_102/usr/local/cuda \
// RUN: --offload-arch=sm_52 -nogpulibc -nogpuinc %s 2>&1 \
// RUN: | FileCheck --check-prefix=NO-LIBC-GPU %s
-// NO-LIBC-GPU-NOT: "-lcgpu"{{.*}}"-lmgpu"
+// NO-LIBC-GPU-NOT: -lmgpu{{.*}}-lcgpu
diff --git a/libc/CMakeLists.txt b/libc/CMakeLists.txt
index 3d775736616745..19f0f4de5f232e 100644
--- a/libc/CMakeLists.txt
+++ b/libc/CMakeLists.txt
@@ -43,7 +43,7 @@ set(LIBC_NAMESPACE "__llvm_libc_${LLVM_VERSION_MAJOR}_${LLVM_VERSION_MINOR}_${LL
CACHE STRING "The namespace to use to enclose internal implementations. Must start with '__llvm_libc'."
)
-if(LLVM_LIBC_FULL_BUILD OR LIBC_GPU_BUILD OR LIBC_GPU_ARCHITECTURES)
+if(LLVM_LIBC_FULL_BUILD OR LLVM_LIBC_GPU_BUILD)
if(NOT LIBC_HDRGEN_EXE)
# We need to set up hdrgen first since other targets depend on it.
add_subdirectory(utils/LibcTableGenUtil)
@@ -65,7 +65,7 @@ if(("libc" IN_LIST LLVM_ENABLE_RUNTIMES AND NOT LLVM_RUNTIMES_BUILD) OR
# to build libc-hdrgen and return.
# Always make the RPC server availible to other projects for GPU mode.
- if(LIBC_GPU_BUILD OR LIBC_GPU_ARCHITECTURES)
+ if(LLVM_LIBC_GPU_BUILD)
add_subdirectory(utils/gpu/server)
endif()
return()
@@ -105,7 +105,7 @@ if(COMMAND_RETURN_CODE EQUAL 0)
message(STATUS "Set COMPILER_RESOURCE_DIR to "
"${COMPILER_RESOURCE_DIR} using --print-resource-dir")
else()
- if (LIBC_TARGET_ARCHITECTURE_IS_GPU)
+ if (LIBC_TARGET_OS_IS_GPU)
message(FATAL_ERROR "COMPILER_RESOURCE_DIR must be set for GPU builds")
else()
set(COMPILER_RESOURCE_DIR OFF)
@@ -203,11 +203,7 @@ foreach(config_path IN LISTS LIBC_CONFIG_JSON_FILE_LIST)
load_libc_config(${config_path}/config.json ${cmd_line_conf})
endforeach()
-if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
- set(LIBC_INCLUDE_DIR ${CMAKE_CURRENT_BINARY_DIR}/include)
- set(LIBC_INSTALL_INCLUDE_DIR ${CMAKE_INSTALL_INCLUDEDIR}/gpu-none-llvm)
- set(LIBC_LIBRARY_DIR ${LLVM_LIBRARY_OUTPUT_INTDIR})
-elseif(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND LIBC_ENABLE_USE_BY_CLANG)
+if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND LIBC_ENABLE_USE_BY_CLANG)
set(LIBC_INCLUDE_DIR ${LLVM_BINARY_DIR}/include/${LLVM_DEFAULT_TARGET_TRIPLE})
set(LIBC_INSTALL_INCLUDE_DIR ${CMAKE_INSTALL_INCLUDEDIR}/${LLVM_DEFAULT_TARGET_TRIPLE})
set(LIBC_LIBRARY_DIR ${LLVM_LIBRARY_OUTPUT_INTDIR}/${LLVM_DEFAULT_TARGET_TRIPLE})
@@ -222,10 +218,14 @@ else()
set(LIBC_INCLUDE_DIR ${CMAKE_BINARY_DIR}/include)
set(LIBC_LIBRARY_DIR ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
endif()
- set(LIBC_INSTALL_INCLUDE_DIR ${CMAKE_INSTALL_INCLUDEDIR})
+ if(LIBC_TARGET_OS_IS_GPU)
+ set(LIBC_INSTALL_INCLUDE_DIR ${CMAKE_INSTALL_INCLUDEDIR}/${LLVM_DEFAULT_TARGET_TRIPLE})
+ else()
+ set(LIBC_INSTALL_INCLUDE_DIR ${CMAKE_INSTALL_INCLUDEDIR})
+ endif()
endif()
-if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+if(LIBC_TARGET_OS_IS_GPU)
include(prepare_libc_gpu_build)
set(LIBC_ENABLE_UNITTESTS OFF)
endif()
diff --git a/libc/cmake/modules/LLVMLibCArchitectures.cmake b/libc/cmake/modules/LLVMLibCArchitectures.cmake
index 623ed774be7270..d9334608c99f07 100644
--- a/libc/cmake/modules/LLVMLibCArchitectures.cmake
+++ b/libc/cmake/modules/LLVMLibCArchitectures.cmake
@@ -6,18 +6,6 @@
# platform.
# ------------------------------------------------------------------------------
-if(LIBC_GPU_BUILD OR LIBC_GPU_ARCHITECTURES)
- # We set the generic target and OS to "gpu" here. More specific defintions
- # for the exact target GPU are set up in prepare_libc_gpu_build.cmake.
- set(LIBC_TARGET_OS "gpu")
- set(LIBC_TARGET_ARCHITECTURE_IS_GPU TRUE)
- set(LIBC_TARGET_ARCHITECTURE "gpu")
- if(LIBC_TARGET_TRIPLE)
- message(WARNING "LIBC_TARGET_TRIPLE is ignored as LIBC_GPU_BUILD is on. ")
- endif()
- return()
-endif()
-
if(MSVC)
# If the compiler is visual c++ or equivalent, we will assume a host build.
set(LIBC_TARGET_OS ${CMAKE_HOST_SYSTEM_NAME})
@@ -59,6 +47,10 @@ function(get_arch_and_system_from_triple triple arch_var sys_var)
set(target_arch "riscv32")
elseif(target_arch MATCHES "^riscv64")
set(target_arch "riscv64")
+ elseif(target_arch MATCHES "^amdgcn")
+ set(target_arch "amdgpu")
+ elseif(target_arch MATCHES "^nvptx64")
+ set(target_arch "nvptx")
else()
return()
endif()
@@ -75,6 +67,12 @@ function(get_arch_and_system_from_triple triple arch_var sys_var)
set(target_sys "darwin")
endif()
+ # Setting OS name for GPU architectures.
+ list(GET triple_comps -1 target_sys)
+ if(target_sys MATCHES "^amdhsa" OR target_sys MATCHES "^cuda")
+ set(target_sys "gpu")
+ endif()
+
set(${sys_var} ${target_sys} PARENT_SCOPE)
endfunction(get_arch_and_system_from_triple)
@@ -156,6 +154,10 @@ elseif(LIBC_TARGET_ARCHITECTURE STREQUAL "riscv64")
elseif(LIBC_TARGET_ARCHITECTURE STREQUAL "riscv32")
set(LIBC_TARGET_ARCHITECTURE_IS_RISCV32 TRUE)
set(LIBC_TARGET_ARCHITECTURE "riscv")
+elseif(LIBC_TARGET_ARCHITECTURE STREQUAL "amdgpu")
+ set(LIBC_TARGET_ARCHITECTURE_IS_AMDGPU TRUE)
+elseif(LIBC_TARGET_ARCHITECTURE STREQUAL "nvptx")
+ set(LIBC_TARGET_ARCHITECTURE_IS_NVPTX TRUE)
else()
message(FATAL_ERROR
"Unsupported libc target architecture ${LIBC_TARGET_ARCHITECTURE}")
@@ -178,6 +180,8 @@ elseif(LIBC_TARGET_OS STREQUAL "darwin")
set(LIBC_TARGET_OS_IS_DARWIN TRUE)
elseif(LIBC_TARGET_OS STREQUAL "windows")
set(LIBC_TARGET_OS_IS_WINDOWS TRUE)
+elseif(LIBC_TARGET_OS STREQUAL "gpu")
+ set(LIBC_TARGET_OS_IS_GPU TRUE)
else()
message(FATAL_ERROR
"Unsupported libc target operating system ${LIBC_TARGET_OS}")
diff --git a/libc/cmake/modules/LLVMLibCCheckMPFR.cmake b/libc/cmake/modules/LLVMLibCCheckMPFR.cmake
index 9e361f5fd81128..bbaeb9f0dc053f 100644
--- a/libc/cmake/modules/LLVMLibCCheckMPFR.cmake
+++ b/libc/cmake/modules/LLVMLibCCheckMPFR.cmake
@@ -2,7 +2,7 @@ set(LLVM_LIBC_MPFR_INSTALL_PATH "" CACHE PATH "Path to where MPFR is installed (
if(LLVM_LIBC_MPFR_INSTALL_PATH)
set(LIBC_TESTS_CAN_USE_MPFR TRUE)
-elseif(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+elseif(LIBC_TARGET_OS_IS_GPU)
set(LIBC_TESTS_CAN_USE_MPFR FALSE)
else()
try_compile(
diff --git a/libc/cmake/modules/LLVMLibCHeaderRules.cmake b/libc/cmake/modules/LLVMLibCHeaderRules.cmake
index 9e9b598721ab37..19515b1cbcc185 100644
--- a/libc/cmake/modules/LLVMLibCHeaderRules.cmake
+++ b/libc/cmake/modules/LLVMLibCHeaderRules.cmake
@@ -139,7 +139,7 @@ function(add_gen_header target_name)
${hdrgen_deps}
)
- if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+ if(LIBC_TARGET_OS_IS_GPU)
file(MAKE_DIRECTORY ${LIBC_INCLUDE_DIR}/llvm-libc-decls)
set(decl_out_file ${LIBC_INCLUDE_DIR}/llvm-libc-decls/${relative_path})
add_custom_command(
diff --git a/libc/cmake/modules/LLVMLibCLibraryRules.cmake b/libc/cmake/modules/LLVMLibCLibraryRules.cmake
index 81c207ec23176a..1d9c9061a98fd5 100644
--- a/libc/cmake/modules/LLVMLibCLibraryRules.cmake
+++ b/libc/cmake/modules/LLVMLibCLibraryRules.cmake
@@ -50,31 +50,9 @@ function(collect_object_file_deps target result)
endif()
endfunction(collect_object_file_deps)
-# A rule to build a library from a collection of entrypoint objects.
-# Usage:
-# add_entrypoint_library(
-# DEPENDS <list of add_entrypoint_object targets>
-# )
-#
-# NOTE: If one wants an entrypoint to be available in a library, then they will
-# have to list the entrypoint target explicitly in the DEPENDS list. Implicit
-# entrypoint dependencies will not be added to the library.
-function(add_entrypoint_library target_name)
- cmake_parse_arguments(
- "ENTRYPOINT_LIBRARY"
- "" # No optional arguments
- "" # No single value arguments
- "DEPENDS" # Multi-value arguments
- ${ARGN}
- )
- if(NOT ENTRYPOINT_LIBRARY_DEPENDS)
- message(FATAL_ERROR "'add_entrypoint_library' target requires a DEPENDS list "
- "of 'add_entrypoint_object' targets.")
- endif()
-
- get_fq_deps_list(fq_deps_list ${ENTRYPOINT_LIBRARY_DEPENDS})
+function(get_all_object_file_deps result fq_deps_list)
set(all_deps "")
- foreach(dep IN LISTS fq_deps_list)
+ foreach(dep ${fq_deps_list})
get_target_property(dep_type ${dep} "TARGET_TYPE")
if(NOT ((${dep_type} STREQUAL ${ENTRYPOINT_OBJ_TARGET_TYPE}) OR
(${dep_type} STREQUAL ${ENTRYPOINT_EXT_TARGET_TYPE}) OR
@@ -102,6 +80,121 @@ function(add_entrypoint_library target_name)
list(APPEND all_deps ${entrypoint_target})
endforeach(dep)
list(REMOVE_DUPLICATES all_deps)
+ set(${result} ${all_deps} PARENT_SCOPE)
+endfunction()
+
+# A rule to build a library from a collection of entrypoint objects and bundle
+# it into a GPU fatbinary. Usage is the same as 'add_entrypoint_library'.
+# Usage:
+# add_gpu_entrypoint_library(
+# DEPENDS <list of add_entrypoint_object targets>
+# )
+function(add_gpu_entrypoint_library target_name)
+ cmake_parse_arguments(
+ "ENTRYPOINT_LIBRARY"
+ "" # No optional arguments
+ "" # No single value arguments
+ "DEPENDS" # Multi-value arguments
+ ${ARGN}
+ )
+ if(NOT ENTRYPOINT_LIBRARY_DEPENDS)
+ message(FATAL_ERROR "'add_entrypoint_library' target requires a DEPENDS list "
+ "of 'add_entrypoint_object' targets.")
+ endif()
+
+ get_fq_deps_list(fq_deps_list ${ENTRYPOINT_LIBRARY_DEPENDS})
+ get_all_object_file_deps(all_deps "${fq_deps_list}")
+
+ # The GPU 'libc' needs to be exported in a format that can be linked with
+ # offloading langauges like OpenMP or CUDA. This wraps every GPU object into a
+ # fat binary and adds them to a static library.
+ set(objects "")
+ foreach(dep IN LISTS all_deps)
+ set(object $<$<STREQUAL:$<TARGET_NAME_IF_EXISTS:${dep}>,${dep}>:$<TARGET_OBJECTS:${dep}>>)
+ string(FIND ${dep} "." last_dot_loc REVERSE)
+ math(EXPR name_loc "${last_dot_loc} + 1")
+ string(SUBSTRING ${dep} ${name_loc} -1 name)
+ if(LIBC_TARGET_ARCHITECTURE_IS_NVPTX)
+ set(prefix --image=arch=generic,triple=nvptx64-nvidia-cuda,feature=+ptx63)
+ else()
+ set(prefix --image=arch=generic,triple=amdgcn-amd-amdhsa)
+ endif()
+
+ # Use the 'clang-offload-packager' to merge these files into a binary blob.
+ add_custom_command(
+ OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/binary/${name}.gpubin"
+ COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_CURRENT_BINARY_DIR}/binary
+ COMMAND ${LIBC_CLANG_OFFLOAD_PACKAGER}
+ "${prefix},file=$<JOIN:${object},,file=>" -o
+ ${CMAKE_CURRENT_BINARY_DIR}/binary/${name}.gpubin
+ DEPENDS ${dep}
+ COMMENT "Packaging LLVM offloading binary for '${object}'"
+ )
+ add_custom_target(${dep}.__gpubin__ DEPENDS ${dep}
+ "${CMAKE_CURRENT_BINARY_DIR}/binary/${name}.gpubin")
+
+ # CMake does not permit setting the name on object files. In order to have
+ # human readable names we create an empty stub file with the entrypoint
+ # name. This empty file will then have the created binary blob embedded.
+ add_custom_command(
+ OUTPUT "${C...
[truncated]
|
| @@ -533,15 +542,14 @@ endfunction(add_integration_test) | |||
| set(LIBC_HERMETIC_TEST_COMPILE_OPTIONS ${LIBC_COMPILE_OPTIONS_DEFAULT} | |||
There was a problem hiding this comment.
this section was moved into LLVMLibcCCompileOptionRules.cmake in #81917
There was a problem hiding this comment.
Yeah there's a lot of logic that's moved and broken after rebasing. Trying to figure out what's changed.
There was a problem hiding this comment.
it should be just the part here and the part relating to compile options in libc object rules
jhuber6
force-pushed
the
GPURework
branch
3 times, most recently
from
d0f782f to
3c4a7ea
Compare
jhuber6
force-pushed
the
GPURework
branch
2 times, most recently
from
c4758c8 to
63e4205
Compare
|
Tested this one a few machines and it works as expected after some final tweaks. FYI @jplehr and @Artem-B, this will change the CMake configuration required for building and testing on the build bots. The new expected way to test each one respectively would be the following I think specifically there are two CMake issues that might make this annoying. I would like to be able to just do |
jplehr
reviewed
Feb 19, 2024
| string(SUBSTRING ${dep} ${name_loc} -1 name) | ||
| if(LIBC_TARGET_ARCHITECTURE_IS_NVPTX) | ||
| set(prefix --image=arch=generic,triple=nvptx64-nvidia-cuda,feature=+ptx63) | ||
| else() |
There was a problem hiding this comment.
Other places do elseif(LIBC_TARGET_ARCHITECTURE_IS_AMDGPU). Maybe here as well for consistency?
jhuber6
force-pushed
the
GPURework
branch
3 times, most recently
from
8727a96 to
f301308
Compare
| endif() | ||
|
|
||
| get_fq_deps_list(fq_deps_list ${ENTRYPOINT_LIBRARY_DEPENDS}) | ||
| function(get_all_object_file_deps result fq_deps_list) |
There was a problem hiding this comment.
This looks like factoring some existing code into a function - if you landed that refactor without changing what the code does, I think it would make this diff much more legible as the current and new code could align
There was a problem hiding this comment.
It used to be simpler, but I had to rebase it on top of other changes in the libc. I could potentially make a simpler patch beforehand.
|
One large patch may be necessary - is it also necessary to interleave reordering files with changing the contents? It makes the GUI diff tool we're using here essentially useless. If the moving code between files and factoring into functions was a separate commit we'd have a much better chance of seeing what you've changed vs what you've moved. |
|
OK, worked through this patch now. The noise is substantial but it's an improvement on what we have - the overall impression is that the cmake was originally very special cased for GPUs and now treats them very similarly to other targets, with some careful footwork around compiling the loaders for the host. This is also a step towards making the GPU parts of the cmake easier to edit for non-GPU people which is a strong win. Thank you Michael and JP. Solid effort working through this Joseph. Let's ship it. |
JonChesterfield
previously approved these changes
Feb 20, 2024
There was a problem hiding this comment.
Stalled on #81557, trying to remove the approve mark as otherwise i'll forget about this
jhuber6
added a commit
to jhuber6/llvm-project
that referenced
this pull request
Feb 21, 2024
Summary: One problem with using the standard `LLVM_RUNTIME_TARGETS` is that it requires setting 'default' manually. This is very good for use-cases where the user wants to completely override their build, but it is bad for builds where we want to augment the normal list. This is because it requires changing things other than the projects that require it. This patch introduces `LLVM_EXTRA_RUNTIME_TARGETS` which is only expected to append to the default list. This patch also fixes the use-case of `libc` using this new potentially different list. This unblocks llvm#81921 so hopefully this is less contentious and can be merged soon.
jhuber6
force-pushed
the
GPURework
branch
2 times, most recently
from
2cf6f18 to
575390d
Compare
Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required llvm#81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on llvm#81557
|
@jhuber6 ,
there are a lot of CMake error messages started with would you take care of it? |
I'll look into it, my guess is that I used |
|
@vvereschaka Should be fixed now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This is a massive patch because it reworks the entire build and
everything that depends on it. This is not split up because various bots
would fail otherwise. I will attempt to describe the necessary changes
here.
This patch completely reworks how the GPU build is built and targeted.
Previously, we used a standard runtimes build and handled both NVPTX and
AMDGPU in a single build via multi-targeting. This added a lot of
divergence in the build system and prevented us from doing various
things like building for the CPU / GPU at the same time, or exporting
the startup libraries or running tests without a full rebuild.
The new appraoch is to handle the GPU builds as strict cross-compiling
runtimes. The first step required
#81557 to allow the
LIBCtarget to build for the GPU without touching the other targets. This
means that the GPU uses all the same handling as the other builds in
libc.The new expected way to build the GPU libc is with
LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda.The second step was reworking how we generated the embedded GPU library
by moving it into the library install step. Where we previously had one
libcgpu.awe now havelibcgpu-amdgpu.aandlibcgpu-nvptx.a. Thispatch includes the necessary clang / OpenMP changes to make that not
break the bots when this lands.
We unfortunately still require that the NVPTX target has an
internaltarget for tests. This is because the NVPTX target needs to do LTO for
the provided version (The offloading toolchain can handle it) but cannot
use it for the native toolchain which is used for making tests.
This approach is vastly superior in every way, allowing us to treat the
GPU as a standard cross-compiling target. We can now install the GPU
utilities to do things like use the offload tests and other fun things.
Some certain utilities need to be built with
--target=${LLVM_HOST_TRIPLE}as well. I think this is a fine workaround as wewill always assume that the GPU
libcis a cross-build with a functioning host.Depends on #81557