Skip to content

Commit

Permalink
Automate updating third-party URLs, and support Clang 7 ASAN/TSAN bui…
Browse files Browse the repository at this point in the history
…lds without Linuxbrew

Summary:
- Remove per-build-type .txt files with third-party URLs. Replace them all with one file, thirdparty_archives.yml.
- Introduce a Python-based tool called thirdparty_tool that allows to update thirdparty_archives.yml based on the thirdparty archives available at https://github.com/yugabyte/yugabyte-db-thirdparty/releases. The latest archive with the branch specifier consistent with the current YugabyteDB version (as determined by version.txt) is selected.
- Adding a jenkins_jobs.yml file that determines the set of jobs to run in the branch. Needs changes from D11619.
- Support for the "clang7" compiler type, which does not rely on Linuxbrew. We do not build LLVM 7 with the Linuxbrew-based thirdparty package anymore so we have to transition to this new mode for running ASAN/TSAN. We will gradually upgrade the Clang version we use for ASAN/TSAN.
- Do not use a "clang_uninstrumented" directory. It was only needed when we had both GCC-compiled and Clang-compiled code in the same thirdparty package.

Test Plan:
Jenkins: test regex: .*(fs_manager-test|TestPgInsert).*

Test regex is there to ensure we can run a C++ test and a Java test (to test the testing infrastructure) without running all the tests.

Reviewers: bogdan, steve.varnau, oleg

Reviewed By: steve.varnau, oleg

Subscribers: oleg, jenkins-bot, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D11599
  • Loading branch information
mbautin committed May 27, 2021
1 parent 1cb456a commit d414500
Show file tree
Hide file tree
Showing 30 changed files with 916 additions and 254 deletions.
19 changes: 2 additions & 17 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -220,15 +220,7 @@ endif()
if ("${YB_BUILD_TYPE}" MATCHES "^(asan|tsan)$")
set(THIRDPARTY_INSTRUMENTATION_TYPE "${YB_BUILD_TYPE}")
elseif ("${COMPILER_FAMILY}" STREQUAL "clang")
if(NOT APPLE AND
(NOT EXISTS "${YB_THIRDPARTY_DIR}/installed/clang_uninstrumented/libcxx" OR
"${COMPILER_VERSION}" VERSION_GREATER_EQUAL "10.0.0"))
# This must be a Clang third-party build where we use "uninstrumented" instead of
# "clang_uninstrumented" because everything is built with Clang.
set(THIRDPARTY_INSTRUMENTATION_TYPE "uninstrumented")
else()
set(THIRDPARTY_INSTRUMENTATION_TYPE "clang_uninstrumented")
endif()
set(THIRDPARTY_INSTRUMENTATION_TYPE "uninstrumented")
elseif ("${COMPILER_FAMILY}" STREQUAL "gcc")
set(THIRDPARTY_INSTRUMENTATION_TYPE "uninstrumented")
else()
Expand Down Expand Up @@ -507,14 +499,7 @@ set(YB_PREFIX_COMMON "${YB_THIRDPARTY_DIR}/installed/common")
if ("${YB_BUILD_TYPE}" MATCHES "^(asan|tsan)$")
YB_SETUP_SANITIZER()
elseif (NOT APPLE AND IS_CLANG)
# Special setup for Clang on Linux. This is not needed on macOS.
if (EXISTS "${YB_THIRDPARTY_DIR}/installed/clang_uninstrumented/libcxx")
YB_SETUP_CLANG("clang_uninstrumented")
else()
# Going forward, for third-party dependency builds where we are only using Clang to build
# everything, we will simply use the "uninstrumented" path instead.
YB_SETUP_CLANG("uninstrumented")
endif()
YB_SETUP_CLANG()
endif()

if (USING_LINUXBREW)
Expand Down
Loading

0 comments on commit d414500

Please sign in to comment.