[SYCL][UR] Update CMake to enable easier local dev (intel#11492)

This patch introduces new ways to include the Unified Runtime source code in the CMake build, the default behaviour is unchanged. It introduces three new CMake variables: * `SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_REPO` is a variable which can be used to override the `UNIFIED_RUNTIME_REPO` variable used by `FetchContent` to attain the Unified Runtime source code. * `SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_TAG` is a variable which can be used to override the `UNIFIED_RUNTIME_TAG` variable used by `FetchContent` to attain the Unified Runtime source code. * `SYCL_PI_UR_USE_FETCH_CONTENT` is an option to control if CMake should use `FetchContent` to pull in the Unified Runtime repository, it defaults to `ON`. When set to `OFF`, `FetchContent` will not be used, instead: * The path specified by variable `SYCL_PI_UR_SOURCE_DIR` will be used with `add_directory()`. This can be used to point at an adjacent directory containing a clone of the Unified Runtime repository. * The path `sycl/plugins/unified_runtime/unified-runtime` will be used, if it exists. This can be used as-if an in-tree build. * `SYCL_PI_UR_SOURCE_DIR` is a variable used to specify the path to the Unified Runtime repository when `SYCL_PI_UR_USE_FETCH_CONTENT` is set of `OFF`. Additionally: * The logic using `UNIFIED_RUNTIME_LIBRARY` and `UNIFIED_RUNTIME_INCLUDE_DIR` was removed because `UNIFIED_RUNTIME_LIBRARY` has no other uses and the `if` statement would always be true. * The `unified_runtime` PI plugin is now always included in the CMake build.
mmoadeli · Oct 18, 2023 · 139b131 · 139b131
1 parent abe3efb
commit 139b131
Show file tree

Hide file tree

Showing 3 changed files with 145 additions and 63 deletions.
diff --git a/sycl/doc/GetStartedGuide.md b/sycl/doc/GetStartedGuide.md
@@ -14,6 +14,7 @@ and a wide range of compute accelerators such as GPU and FPGA.
     * [Build DPC++ toolchain with support for HIP NVIDIA](#build-dpc-toolchain-with-support-for-hip-nvidia)
     * [Build DPC++ toolchain with support for ESIMD CPU Emulation](#build-dpc-toolchain-with-support-for-esimd-cpu-emulation)
     * [Build DPC++ toolchain with support for runtime kernel fusion](#build-dpc-toolchain-with-support-for-runtime-kernel-fusion)
+    * [Build DPC++ toolchain with a custom Unified Runtime](#build-dpc-toolchain-with-a-custom-unified-runtime)
     * [Build Doxygen documentation](#build-doxygen-documentation)
     * [Deployment](#deployment)
   * [Use DPC++ toolchain](#use-dpc-toolchain)
@@ -302,15 +303,15 @@ package will be generated from source codes downloaded from its open source
 project and installed in your deploy directory during toolchain build.
 
 To enable support for ESIMD CPU emulation, follow the instructions for the Linux
-DPC++ toolchain, but add the `--enable-esimd-emulator'.
+DPC++ toolchain, but add the `--enable-esimd-emulator`.
 
 Enabling this flag requires following packages installed.
 
 * Ubuntu 22.04
   * libva-dev / 2.7.0-2
   * libffi-dev / 3.3-4
   * libtool
-* RHEL 8.*
+* RHEL 8.\*
   * libffi
   * libffi-devel
   * libva
@@ -348,6 +349,36 @@ After CMake cache is generated, build the documentation with `doxygen-sycl`
 target. It will be put to `$DPCPP_HOME/llvm/build/tools/sycl/doc/html`
 directory.
 
+### Build DPC++ toolchain with a custom Unified Runtime
+
+DPC++ uses the [Unified Runtime](https://github.com/oneapi-src/unified-runtime)
+under the hood to provide implementations of various SYCL backends. By default
+the source code for the Unified Runtime will be acquired using CMake's
+[FetchCotent](https://cmake.org/cmake/help/latest/module/FetchContent.html). The
+specific repository URL and revision tag used can be found in the file
+`sycl/plugins/unified_runtime/CMakeLists.txt` searching for the variables
+`UNIFIED_RUNTIME_REPO` and `UNIFIED_RUNTIME_TAG`.
+
+In order to enable developers, a number of CMake variables are available to
+control which revision of Unified Runtime should be used when building DPC++:
+
+* `SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_REPO` is a variable which can be used to
+  override the `UNIFIED_RUNTIME_REPO` variable used by `FetchContent` to attain
+  the Unified Runtime source code.
+* `SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_TAG` is a variable which can be used to
+  override the `UNIFIED_RUNTIME_TAG` variable used by `FetchContent` to attain
+  the Unified Runtime source code.
+* `SYCL_PI_UR_USE_FETCH_CONTENT` is an option to control if CMake should use
+  `FetchContent` to pull in the Unified Runtime repository, it defaults to `ON`.
+  When set to `OFF`, `FetchContent` will not be used, instead:
+  * The path specified by variable `SYCL_PI_UR_SOURCE_DIR` will be used with
+    `add_directory()`. This can be used to point at an adjacent directory
+    containing a clone of the Unified Runtime repository.
+  * The path `sycl/plugins/unified_runtime/unified-runtime` will be used, if it
+    exists. This can be used as-if an in-tree build.
+* `SYCL_PI_UR_SOURCE_DIR` is a variable used to specify the path to the Unified
+  Runtime repository when `SYCL_PI_UR_USE_FETCH_CONTENT` is set of `OFF`.
+
 ### Deployment
 
 TODO: add instructions how to deploy built DPC++ toolchain.

diff --git a/sycl/plugins/CMakeLists.txt b/sycl/plugins/CMakeLists.txt
@@ -2,11 +2,9 @@ if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang|IntelLLVM" )
   set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-covered-switch-default")
 endif()
 
-# level_zero and opencl plugins depend today on unified_runtime plugin
-# and unified_runtime plugin is not an independent plugin, adding it explicitly
-if ("level_zero" IN_LIST SYCL_ENABLE_PLUGINS OR "opencl" IN_LIST SYCL_ENABLE_PLUGINS)
-  add_subdirectory(unified_runtime)
-endif()
+# all plugins on unified_runtime plugin and unified_runtime plugin is not an
+# independent plugin, adding it explicitly
+add_subdirectory(unified_runtime)
 
 foreach(plugin ${SYCL_ENABLE_PLUGINS})
   add_subdirectory(${plugin})

diff --git a/sycl/plugins/unified_runtime/CMakeLists.txt b/sycl/plugins/unified_runtime/CMakeLists.txt
@@ -1,6 +1,53 @@
 # PI Unified Runtime plugin library.
 #
-if (NOT DEFINED UNIFIED_RUNTIME_LIBRARY OR NOT DEFINED UNIFIED_RUNTIME_INCLUDE_DIR)
+
+# Options to override the default behaviour of the FetchContent to include UR
+# source code.
+set(SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_REPO
+  "" CACHE STRING "Override the Unified Runtime FetchContent repository")
+set(SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_TAG
+  "" CACHE STRING "Override the Unified Runtime FetchContent tag")
+
+# Options to disable use of FetchContent to include Unified Runtime source code
+# to improve developer workflow.
+option(SYCL_PI_UR_USE_FETCH_CONTENT
+  "Use FetchContent to acquire the Unified Runtime source code" ON)
+set(SYCL_PI_UR_SOURCE_DIR
+  "" CACHE PATH "Path to root of Unified Runtime repository")
+
+# Override default to enable building tests from unified-runtime
+set(UR_BUILD_TESTS OFF CACHE BOOL "Build unit tests.")
+set(UMF_ENABLE_POOL_TRACKING ON)
+
+if("level_zero" IN_LIST SYCL_ENABLE_PLUGINS)
+  set(UR_BUILD_ADAPTER_L0 ON)
+endif()
+if("cuda" IN_LIST SYCL_ENABLE_PLUGINS)
+  set(UR_BUILD_ADAPTER_CUDA ON)
+endif()
+if("hip" IN_LIST SYCL_ENABLE_PLUGINS)
+  set(UR_BUILD_ADAPTER_HIP ON)
+endif()
+# TODO: Set UR_BUILD_ADAPTER_OPENCL once adapter moved
+# TODO: Set UR_BUILD_ADAPTER_NATIVE_CPU once adapter moved
+
+# Disable errors from warnings while building the UR.
+# And remember origin flags before doing that.
+set(CMAKE_CXX_FLAGS_BAK "${CMAKE_CXX_FLAGS}")
+if(WIN32)
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /WX-")
+  set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /WX-")
+  # FIXME: Unified runtime build fails with /DUNICODE
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /UUNICODE")
+  set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /UUNICODE")
+  # USE_Z7 forces use of /Z7 instead of /Zi which is broken with sccache
+  set(USE_Z7 ON)
+else()
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error")
+  set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-error")
+endif()
+
+if(SYCL_PI_UR_USE_FETCH_CONTENT)
   include(FetchContent)
 
   set(UNIFIED_RUNTIME_REPO "https://github.com/oneapi-src/unified-runtime.git")
@@ -12,58 +59,65 @@ if (NOT DEFINED UNIFIED_RUNTIME_LIBRARY OR NOT DEFINED UNIFIED_RUNTIME_INCLUDE_D
   #        [UR][L0] Copy prebuilt L0 to avoid leaking shared folder path
   set(UNIFIED_RUNTIME_TAG b38855ed815ffd076bfde5e5e06170ca4f723dc1)
 
-  if ("level_zero" IN_LIST SYCL_ENABLE_PLUGINS)
-    set(UR_BUILD_ADAPTER_L0 ON)
+  if(SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_REPO)
+    set(UNIFIED_RUNTIME_REPO "${SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_REPO}")
   endif()
-
-  if ("cuda" IN_LIST SYCL_ENABLE_PLUGINS)
-    set(UR_BUILD_ADAPTER_CUDA ON)
+  if(SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_TAG)
+    set(UNIFIED_RUNTIME_TAG "${SYCL_PI_UR_OVERRIDE_FETCH_CONTENT_TAG}")
   endif()
 
-  if ("hip" IN_LIST SYCL_ENABLE_PLUGINS)
-    set(UR_BUILD_ADAPTER_HIP ON)
-  endif()
-  set(UMF_ENABLE_POOL_TRACKING ON)
   message(STATUS "Will fetch Unified Runtime from ${UNIFIED_RUNTIME_REPO}")
   FetchContent_Declare(unified-runtime
     GIT_REPOSITORY    ${UNIFIED_RUNTIME_REPO}
     GIT_TAG           ${UNIFIED_RUNTIME_TAG}
   )
 
-  # Disable errors from warnings while building the UR.
-  # And remember origin flags before doing that.
-  set(CMAKE_CXX_FLAGS_BAK "${CMAKE_CXX_FLAGS}")
-  if(WIN32)
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /WX-")
-    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /WX-")
-    # FIXME: Unified runtime build fails with /DUNICODE
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /UUNICODE")
-    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /UUNICODE")
-    # USE_Z7 forces use of /Z7 instead of /Zi which is broken with sccache
-    set(USE_Z7 ON)
-  else()
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error")
-    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-error")
-  endif()
-
-  # No need to build tests from unified-runtime
-  set(UR_BUILD_TESTS "0" CACHE STRING "0")
-
   FetchContent_GetProperties(unified-runtime)
   FetchContent_MakeAvailable(unified-runtime)
 
-  # Restore original flags
-  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS_BAK}")
-
-  add_library(UnifiedRuntimeLoader ALIAS ur_loader)
-  add_library(UnifiedRuntimeCommon ALIAS ur_common)
-  add_library(UnifiedMallocFramework ALIAS unified_malloc_framework)
+  set(UNIFIED_RUNTIME_SOURCE_DIR
+    "${unified-runtime_SOURCE_DIR}" CACHE PATH
+    "Path to Unified Runtime Headers" FORCE)
+elseif(SYCL_PI_UR_SOURCE_DIR)
+  # SYCL_PI_UR_USE_FETCH_CONTENT is OFF and SYCL_PI_UR_SOURCE_DIR has been set,
+  # use the external Unified Runtime source directory.
+  set(UNIFIED_RUNTIME_SOURCE_DIR
+    "${SYCL_PI_UR_SOURCE_DIR}" CACHE PATH
+    "Path to Unified Runtime Headers" FORCE)
+  add_subdirectory(
+    ${UNIFIED_RUNTIME_SOURCE_DIR}
+    ${CMAKE_CURRENT_BINARY_DIR}/unified-runtime)
+else()
+  # SYCL_PI_UR_USE_FETCH_CONTENT is OFF and SYCL_PI_UR_SOURCE_DIR has not been
+  # set, check if the fallback local directory exists.
+  if(NOT ${CMAKE_CURRENT_SOURCE_DIR}/unified-runtime)
+    message(FATAL_ERROR
+      "SYCL_PI_UR_USE_FETCH_CONTENT is disabled but no alternative Unified
+      Runtime source directory has be proivided, either:
 
+      * Set -DSYCL_PI_UR_SOURCE_DIR=/path/to/unified-runtime
+      * Clone the UR repo in ${CMAKE_CURRENT_SOURCE_DIR}/unified-runtime")
+  endif()
+  # The fallback local directory for the Unified Runtime repository has been
+  # found, use it.
   set(UNIFIED_RUNTIME_SOURCE_DIR
-    ${unified-runtime_SOURCE_DIR} CACHE PATH "Path to Unified Runtime Headers")
-  set(UNIFIED_RUNTIME_INCLUDE_DIR "${UNIFIED_RUNTIME_SOURCE_DIR}/include")
+    "${CMAKE_CURRENT_SOURCE_DIR}/unified-runtime" CACHE PATH
+    "Path to Unified Runtime Headers" FORCE)
+  add_subdirectory(${UNIFIED_RUNTIME_SOURCE_DIR})
 endif()
 
+# Restore original flags
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS_BAK}")
+
+message(STATUS
+  "Using Unified Runtime source directory: ${UNIFIED_RUNTIME_SOURCE_DIR}")
+
+set(UNIFIED_RUNTIME_INCLUDE_DIR "${UNIFIED_RUNTIME_SOURCE_DIR}/include")
+
+add_library(UnifiedRuntimeLoader ALIAS ur_loader)
+add_library(UnifiedRuntimeCommon ALIAS ur_common)
+add_library(UnifiedMallocFramework ALIAS unified_malloc_framework)
+
 add_library(UnifiedRuntime-Headers INTERFACE)
 
 target_include_directories(UnifiedRuntime-Headers
@@ -92,27 +146,15 @@ set(UNIFIED_RUNTIME_PLUGIN_ARGS
 )
 
 # We need for #include <ze_api.h> in common.h
-if ("level_zero" IN_LIST SYCL_ENABLE_PLUGINS)
-    list(APPEND UNIFIED_RUNTIME_PLUGIN_ARGS LevelZeroLoader-Headers)
-endif()
-
-if ("opencl" IN_LIST SYCL_ENABLE_PLUGINS)
-    list(APPEND UNIFIED_RUNTIME_PLUGIN_ARGS OpenCL-ICD)
-endif()
-
-add_sycl_plugin(unified_runtime ${UNIFIED_RUNTIME_PLUGIN_ARGS})
-
 if("level_zero" IN_LIST SYCL_ENABLE_PLUGINS)
-  add_dependencies(sycl-runtime-libraries ur_adapter_level_zero)
+  list(APPEND UNIFIED_RUNTIME_PLUGIN_ARGS LevelZeroLoader-Headers)
 endif()
 
-if("hip" IN_LIST SYCL_ENABLE_PLUGINS)
-  add_dependencies(sycl-runtime-libraries ur_adapter_hip)
+if("opencl" IN_LIST SYCL_ENABLE_PLUGINS)
+  list(APPEND UNIFIED_RUNTIME_PLUGIN_ARGS OpenCL-ICD)
 endif()
 
-if ("cuda" IN_LIST SYCL_ENABLE_PLUGINS)
-  add_dependencies(sycl-runtime-libraries ur_adapter_cuda)
-endif()
+add_sycl_plugin(unified_runtime ${UNIFIED_RUNTIME_PLUGIN_ARGS})
 
 if ("opencl" IN_LIST SYCL_ENABLE_PLUGINS)
   # Build OpenCL adapter
@@ -207,7 +249,7 @@ if("native_cpu" IN_LIST SYCL_ENABLE_PLUGINS)
 endif()
 
 
-if (TARGET UnifiedRuntimeLoader)
+if(TARGET UnifiedRuntimeLoader)
   set_target_properties(hello_world PROPERTIES EXCLUDE_FROM_ALL 1 EXCLUDE_FROM_DEFAULT_BUILD 1)
   # Install the UR loader.
   # TODO: this is piggy-backing on the existing target component level-zero-sycl-dev
@@ -221,10 +263,21 @@ if (TARGET UnifiedRuntimeLoader)
 endif()
 
 # Install the UR adapters too
-if ("level_zero" IN_LIST SYCL_ENABLE_PLUGINS)
+if("level_zero" IN_LIST SYCL_ENABLE_PLUGINS)
+  add_dependencies(sycl-runtime-libraries ur_adapter_level_zero)
+
+  # Install the UR adapters too
   install(TARGETS ur_adapter_level_zero
     LIBRARY DESTINATION "lib${LLVM_LIBDIR_SUFFIX}" COMPONENT level-zero-sycl-dev
     ARCHIVE DESTINATION "lib${LLVM_LIBDIR_SUFFIX}" COMPONENT level-zero-sycl-dev
     RUNTIME DESTINATION "bin" COMPONENT level-zero-sycl-dev
-)
+  )
+endif()
+
+if("cuda" IN_LIST SYCL_ENABLE_PLUGINS)
+  add_dependencies(sycl-runtime-libraries ur_adapter_cuda)
+endif()
+
+if("hip" IN_LIST SYCL_ENABLE_PLUGINS)
+  add_dependencies(sycl-runtime-libraries ur_adapter_hip)
 endif()