Skip to content

[SYCL] Avoid ABI issues with SYCL RT #685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 16, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions clang/lib/Driver/ToolChains/MSVC.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -354,8 +354,13 @@ void visualstudio::Linker::ConstructJob(Compilation &C, const JobAction &JA,
!C.getDriver().IsCLMode())
CmdArgs.push_back("-defaultlib:libcmt");

if (!Args.hasArg(options::OPT_nostdlib) && Args.hasArg(options::OPT_fsycl))
CmdArgs.push_back("-defaultlib:sycl.lib");
if (!Args.hasArg(options::OPT_nostdlib) && Args.hasArg(options::OPT_fsycl)) {
if (Args.hasArg(options::OPT__SLASH_MDd) ||
Args.hasArg(options::OPT__SLASH_MTd))
CmdArgs.push_back("-defaultlib:sycld.lib");
else
CmdArgs.push_back("-defaultlib:sycl.lib");
}

for (const auto *A : Args.filtered(options::OPT_foffload_static_lib_EQ))
CmdArgs.push_back(
Expand Down
6 changes: 6 additions & 0 deletions clang/test/Driver/sycl-offload.c
Original file line number Diff line number Diff line change
Expand Up @@ -500,6 +500,12 @@
// CHECK-LINK-SYCL: "{{.*}}link{{(.exe)?}}"
// CHECK-LINK-SYCL: "-defaultlib:sycl.lib"

/// Check sycld.lib is chosen with /MDd and /MTd
// RUN: %clang_cl -fsycl /MDd %s -o %t -### 2>&1 | FileCheck -check-prefix=CHECK-LINK-SYCL-DEBUG %s
// RUN: %clang_cl -fsycl /MTd %s -o %t -### 2>&1 | FileCheck -check-prefix=CHECK-LINK-SYCL-DEBUG %s
// CHECK-LINK-SYCL-DEBUG: "{{.*}}link{{(.exe)?}}"
// CHECK-LINK-SYCL-DEBUG: "-defaultlib:sycld.lib"

/// ###########################################################################

/// test behaviors of -foffload-static-lib=<lib>
Expand Down
7 changes: 6 additions & 1 deletion sycl/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -138,12 +138,17 @@ COMMENT "Copying SYCL headers ...")
# Configure SYCL headers
install(DIRECTORY "${sycl_inc_dir}/." DESTINATION "${LLVM_INST_INC_DIRECTORY}" COMPONENT sycl-headers)

set(SYCL_RT_LIBS sycl)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should build two versions of SYCL runtime library. I consider this patch as a work-around for poor SYCL library ABI design.
We didn't pay much attention to the library ABI design and it .
I would prefer you to review binary interface and make it stable and portable instead of this change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bader - Alexey, Do you suggest replacing all standard classes such as vector, etc with something else?
Those debug|release versions of C Runtime Libraries are linked when you use any of this impressive list of C++ header files: https://docs.microsoft.com/en-us/cpp/standard-library/cpp-standard-library-header-files?view=vs-2019

See, it is written here: https://docs.microsoft.com/en-us/cpp/c-runtime-library/crt-library-features?view=vs-2019

When you build a release version of your project, one of the basic C run-time libraries (libcmt.lib, msvcmrt.lib, msvcrt.lib) is linked by default, depending on the compiler option you choose (multithreaded, DLL, /clr). If you include one of the C++ Standard Library header files in your code, a C++ Standard Library will be linked in automatically by Visual C++ at compile time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding to my previous statement.
I believe the poor ABI is not caused by our library implementation, but by the SYCL SPEC.
SYCL SPEC makes us using those std::vector, std::string, etc
Here is extract from it:

namespace cl {
9 namespace sycl {
10
11 template < class T, class Alloc = std::allocator >
12 using vector_class = std::vector<T, Alloc>;
13
14 using string_class = std::string;
15
16 template<class R, class... ArgTypes>
17 using function_class = std::function<R(ArgTypes...)>;
18
19 using mutex_class = std::mutex;
20
21 template
22 using shared_ptr_class = std::shared_ptr;
23
24 template
25 using unique_ptr_class = std::unique_ptr;
26
27 template
28 using weak_ptr_class = std::weak_ptr;
29
30 template
31 using hash_class = std::hash;
32
33 using exception_ptr_class = std::exception_ptr;
34
35 } // sycl
36 } // cl

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alexey, Do you suggest replacing all standard classes such as vector, etc with something else?

Yes.

SYCL SPEC makes us using those std::vector, std::string, etc

No, it's not. It requires SYCL implementation to define <...>_class classes, which can be aliases to standard library classes, but specification does not require this particular implementation.

Anyway we do not have to use SYCL API in the library interface. It's an implementation decision - how functionality is split between library and headers and interface is used between them. Using C++ classes in ABI is not recommended as C++ classes layout is implementation defined, so using different compilers to build headers and a library doesn't work in general case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I understood what you mean, and agree now that it would be great to avoid using std::vector args/returns of exported functions in sycl/source. That would let us have only 1 sycl.dll. Thank you.

if (MSVC)
list(APPEND SYCL_RT_LIBS sycld)
endif()

# SYCL runtime library
add_subdirectory(source)

# SYCL toolchain builds all components: compiler, libraries, headers, etc.
add_custom_target( sycl-toolchain
DEPENDS sycl
DEPENDS ${SYCL_RT_LIBS}
clang
clang-offload-wrapper
clang-offload-bundler
Expand Down
1 change: 1 addition & 0 deletions sycl/doc/GetStartedWithSYCLCompiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@ int main() {
translation units.
- SYCL host device is not fully supported.
- SYCL works only with OpenCL implementations supporting out-of-order queues.
- On Windows linking SYCL applications with `/MTd` flag is known to cause crashes.

# Find More

Expand Down
217 changes: 132 additions & 85 deletions sycl/source/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,101 +4,148 @@
#cmake_policy(SET CMP0057 NEW)
#include(AddLLVM)

add_library(sycl SHARED
"${sycl_inc_dir}/CL/sycl.hpp"
"detail/builtins_common.cpp"
"detail/builtins_geometric.cpp"
"detail/builtins_integer.cpp"
"detail/builtins_math.cpp"
"detail/builtins_relational.cpp"
"detail/pi.cpp"
"detail/pi_opencl.cpp"
"detail/common.cpp"
"detail/context_impl.cpp"
"detail/device_impl.cpp"
"detail/device_info.cpp"
"detail/event_impl.cpp"
"detail/force_device.cpp"
"detail/helpers.cpp"
"detail/image_accessor_util.cpp"
"detail/image_impl.cpp"
"detail/kernel_impl.cpp"
"detail/kernel_info.cpp"
"detail/memory_manager.cpp"
"detail/platform_impl.cpp"
"detail/platform_info.cpp"
"detail/program_impl.cpp"
"detail/program_manager/program_manager.cpp"
"detail/queue_impl.cpp"
"detail/os_util.cpp"
"detail/platform_util.cpp"
"detail/sampler_impl.cpp"
"detail/stream_impl.cpp"
"detail/scheduler/commands.cpp"
"detail/scheduler/scheduler.cpp"
"detail/scheduler/graph_processor.cpp"
"detail/scheduler/graph_builder.cpp"
"detail/usm/clusm.cpp"
"detail/usm/usm_dispatch.cpp"
"detail/usm/usm_impl.cpp"
"detail/util.cpp"
"context.cpp"
"device.cpp"
"device_selector.cpp"
"event.cpp"
"exception.cpp"
"exception_list.cpp"
"half_type.cpp"
"kernel.cpp"
"platform.cpp"
"queue.cpp"
"ordered_queue.cpp"
"sampler.cpp"
"stream.cpp"
"spirv_ops.cpp"
)
function(add_sycl_rt_library LIB_NAME)

add_library(${LIB_NAME} SHARED ${ARGN})

add_dependencies(${LIB_NAME}
ocl-icd
ocl-headers
sycl-headers
)

set_target_properties(${LIB_NAME} PROPERTIES LINKER_LANGUAGE CXX)

if (MSVC)
target_compile_definitions(${LIB_NAME} PRIVATE __SYCL_BUILD_SYCL_DLL )
endif()
target_include_directories(${LIB_NAME} PRIVATE "${sycl_inc_dir}")
target_link_libraries(${LIB_NAME}
PRIVATE OpenCL::Headers
PRIVATE ${OpenCL_LIBRARIES}
)
if (SYCL_USE_LIBCXX)
if ((CMAKE_CXX_COMPILER_ID STREQUAL "GNU") OR
(CMAKE_CXX_COMPILER_ID STREQUAL "Clang"))
target_compile_options(${LIB_NAME} PRIVATE -nostdinc++)
if ((NOT (DEFINED SYCL_LIBCXX_INCLUDE_PATH)) OR (NOT (DEFINED SYCL_LIBCXX_LIBRARY_PATH)))
message(FATAL_ERROR "When building with libc++ SYCL_LIBCXX_INCLUDE_PATHS and"
"SYCL_LIBCXX_LIBRARY_PATH should be set")
endif()
target_include_directories(${LIB_NAME} PRIVATE "${SYCL_LIBCXX_INCLUDE_PATH}")
target_link_libraries(${LIB_NAME} PRIVATE "-L${SYCL_LIBCXX_LIBRARY_PATH}" -nodefaultlibs -lc++ -lc++abi -lm -lc -lgcc_s -lgcc)
else()
message(FATAL_ERROR "Build with libc++ is not yet supported for this compiler")
endif()
else()

# Workaround for bug in GCC version 5 and higher.
# More information https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1568899
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 5.0)
target_link_libraries(${LIB_NAME} PRIVATE gcc_s gcc)
endif()

add_dependencies(sycl
ocl-icd
ocl-headers
sycl-headers
endif()
endfunction(add_sycl_rt_library)

set(SYCL_SOURCES
"${sycl_inc_dir}/CL/sycl.hpp"
"detail/builtins_common.cpp"
"detail/builtins_geometric.cpp"
"detail/builtins_integer.cpp"
"detail/builtins_math.cpp"
"detail/builtins_relational.cpp"
"detail/pi.cpp"
"detail/pi_opencl.cpp"
"detail/common.cpp"
"detail/context_impl.cpp"
"detail/device_impl.cpp"
"detail/device_info.cpp"
"detail/event_impl.cpp"
"detail/force_device.cpp"
"detail/helpers.cpp"
"detail/image_accessor_util.cpp"
"detail/image_impl.cpp"
"detail/kernel_impl.cpp"
"detail/kernel_info.cpp"
"detail/memory_manager.cpp"
"detail/platform_impl.cpp"
"detail/platform_info.cpp"
"detail/program_impl.cpp"
"detail/program_manager/program_manager.cpp"
"detail/queue_impl.cpp"
"detail/os_util.cpp"
"detail/platform_util.cpp"
"detail/sampler_impl.cpp"
"detail/stream_impl.cpp"
"detail/scheduler/commands.cpp"
"detail/scheduler/scheduler.cpp"
"detail/scheduler/graph_processor.cpp"
"detail/scheduler/graph_builder.cpp"
"detail/usm/clusm.cpp"
"detail/usm/usm_dispatch.cpp"
"detail/usm/usm_impl.cpp"
"detail/util.cpp"
"context.cpp"
"device.cpp"
"device_selector.cpp"
"event.cpp"
"exception.cpp"
"exception_list.cpp"
"half_type.cpp"
"kernel.cpp"
"platform.cpp"
"queue.cpp"
"ordered_queue.cpp"
"sampler.cpp"
"stream.cpp"
"spirv_ops.cpp"
)

set_target_properties(sycl PROPERTIES LINKER_LANGUAGE CXX)
add_sycl_rt_library(sycl ${SYCL_SOURCES})

if (MSVC)
target_compile_definitions(sycl PRIVATE __SYCL_BUILD_SYCL_DLL )
endif()
target_include_directories(sycl PRIVATE "${sycl_inc_dir}")
target_link_libraries(sycl
PRIVATE OpenCL::Headers
PRIVATE ${OpenCL_LIBRARIES}
)
if (SYCL_USE_LIBCXX)
if ((CMAKE_CXX_COMPILER_ID STREQUAL "GNU") OR
(CMAKE_CXX_COMPILER_ID STREQUAL "Clang"))
target_compile_options(sycl PRIVATE -nostdinc++)
if ((NOT (DEFINED SYCL_LIBCXX_INCLUDE_PATH)) OR (NOT (DEFINED SYCL_LIBCXX_LIBRARY_PATH)))
message(FATAL_ERROR "When building with libc++ SYCL_LIBCXX_INCLUDE_PATHS and"
"SYCL_LIBCXX_LIBRARY_PATH should be set")
endif()
target_include_directories(sycl PRIVATE "${SYCL_LIBCXX_INCLUDE_PATH}")
target_link_libraries(sycl PRIVATE "-L${SYCL_LIBCXX_LIBRARY_PATH}" -nodefaultlibs -lc++ -lc++abi -lm -lc -lgcc_s -lgcc)
else()
message(FATAL_ERROR "Build with libc++ is not yet supported for this compiler")
# MSVC provides two incompatible build variants for its CRT: release and debug
# To avoid potential issues in user code we also need to provide two kinds
# of SYCL Runtime Library for release and debug configurations.
set(SYCL_CXX_FLAGS "")
if (CMAKE_BUILD_TYPE MATCHES "Debug")
set(SYCL_CXX_FLAGS "${CMAKE_CXX_FLAGS_DEBUG}")
string(REPLACE "/MDd" "" SYCL_CXX_FLAGS "${SYCL_CXX_FLAGS}")
string(REPLACE "/MTd" "" SYCL_CXX_FLAGS "${SYCL_CXX_FLAGS}")
else()
if (CMAKE_BUILD_TYPE MATCHES "Release")
set(SYCL_CXX_FLAGS "${CMAKE_CXX_FLAGS_RELEASE}")
elseif (CMAKE_BUILD_TYPE MATCHES "RelWithDebInfo")
set(SYCL_CXX_FLAGS "${CMAKE_CXX_FLAGS_MINSIZEREL}")
elseif (CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(SYCL_CXX_FLAGS "${CMAKE_CXX_FLAGS_RELWITHDEBINFO}")
endif()
else()
string(REPLACE "/MD" "" SYCL_CXX_FLAGS "${SYCL_CXX_FLAGS}")
string(REPLACE "/MT" "" SYCL_CXX_FLAGS "${SYCL_CXX_FLAGS}")
endif()

# Workaround for bug in GCC version 5 and higher.
# More information https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1568899
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 5.0)
target_link_libraries(sycl PRIVATE gcc_s gcc)
endif()
# target_compile_options requires list of options, not a string
string(REPLACE " " ";" SYCL_CXX_FLAGS "${SYCL_CXX_FLAGS}")

set(SYCL_CXX_FLAGS_RELEASE "${SYCL_CXX_FLAGS};/MD")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can a DLL built with /MD flag be used by a program built with /MT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should work fine.
Actually, I think providing a library variant built with /MT (if that's what your're implying) may cause a lot more trouble. Let's say, we link SYCL RT DLL against CRT of version X. And user downloads pre-built compiler and runtime, develops an application and links it against SYCL RT and CRT of version Y. There are now 2 CRTs in app's address space. And they may be incompatible. Thus, I think providing only /MD version should be sufficient.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think providing a library variant built with /MT (if that's what your're implying)

I did not suggest that. The only question is whether /MT for a user program works with sycl.dll or not. If it works fine, then please add another RUN line with /MT to the tests. If it doesn't - then we should explicitly state this requirement or provide a solution for /MT.

set(SYCL_CXX_FLAGS_DEBUG "${SYCL_CXX_FLAGS};/MDd")

# CMake automatically applies these flags to all targets. To override this
# behavior, options lists are reset.
set(CMAKE_CXX_FLAGS_RELEASE "")
set(CMAKE_CXX_FLAGS_MINSIZEREL "")
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "")
set(CMAKE_CXX_FLAGS_DEBUG "")

target_compile_options(sycl PUBLIC ${SYCL_CXX_FLAGS_RELEASE})

add_sycl_rt_library(sycld ${SYCL_SOURCES})
target_compile_options(sycld PUBLIC ${SYCL_CXX_FLAGS_DEBUG})
endif()

install(TARGETS sycl
install(TARGETS ${SYCL_RT_LIBS}
ARCHIVE DESTINATION "lib" COMPONENT sycl
LIBRARY DESTINATION "lib" COMPONENT sycl
RUNTIME DESTINATION "bin" COMPONENT sycl)
1 change: 1 addition & 0 deletions sycl/test/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ set(LLVM_BUILD_BINARY_DIRS "${LLVM_BINARY_DIR}/bin/")
set(LLVM_TOOLS_DIR "${LLVM_BINARY_DIR}/bin/")
set(CLANG_IN_BUILD "${LLVM_BINARY_DIR}/bin/clang")
set(CLANGXX_IN_BUILD "${LLVM_BINARY_DIR}/bin/clang++")
set(CLANGCL_IN_BUILD "${LLVM_BINARY_DIR}/bin/clang-cl")

get_target_property(SYCL_BINARY_DIR sycl-toolchain BINARY_DIR)
get_target_property(SYCL_SOURCE_DIR sycl-toolchain SOURCE_DIR)
Expand Down
1 change: 1 addition & 0 deletions sycl/test/lit.cfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@

config.substitutions.append( ('%clang_cc1', ' ' + config.clang + ' -cc1 ') )
config.substitutions.append( ('%clangxx', ' ' + config.clangxx + ' -I'+config.opencl_include ) )
config.substitutions.append( ('%clang_cl', ' ' + config.clang_cl + ' /I '+config.opencl_include ) )
config.substitutions.append( ('%clang', ' ' + config.clang + ' -I'+config.opencl_include ) )
config.substitutions.append( ('%llvm_build_libs_dir', config.llvm_build_libs_dir ) )
config.substitutions.append( ('%opencl_include', config.opencl_include ) )
Expand Down
1 change: 1 addition & 0 deletions sycl/test/lit.site.cfg.py.in
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import sys

config.clang = "@CLANG_IN_BUILD@"
config.clangxx = "@CLANGXX_IN_BUILD@"
config.clang_cl = "@CLANGCL_IN_BUILD@"
config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
config.llvm_build_libs_dir = "@LLVM_BUILD_LIBRARY_DIRS@"
Expand Down
46 changes: 46 additions & 0 deletions sycl/test/regression/msvc_crt.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// RUN: %clang_cl -fsycl /MD -o %t1.exe %s
// RUN: %CPU_RUN_PLACEHOLDER %t1.exe
// RUN: %clang_cl -fsycl /MDd -o %t2.exe %s
// RUN: %CPU_RUN_PLACEHOLDER %t2.exe
// RUN: %clang_cl -fsycl /MT -o %t3.exe %s
// RUN: %CPU_RUN_PLACEHOLDER %t3.exe
// REQUIRES: system-windows
//==-------------- msvc_crt.cpp - SYCL MSVC CRT test -----------------------==//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// MSVC provides two different incompatible variants of CRT: debug and release.
// This test checks if clang driver is able to handle this properly.

#include <CL/sycl.hpp>

using namespace cl::sycl;

int main() {
int data[] = {0, 0, 0};

{
buffer<int, 1> b(data, range<1>(3), {property::buffer::use_host_ptr()});
queue q;
q.submit([&](handler &cgh) {
auto B = b.get_access<access::mode::write>(cgh);
cgh.parallel_for<class test>(range<1>(3), [=](id<1> idx) {
B[idx] = 1;
});
});
}

bool isSuccess = true;

for (int i = 0; i < 3; i++)
if (data[i] != 1) isSuccess = false;

if (!isSuccess)
return -1;

return 0;
}