Cannot build `julia-1.9.1-full.tar.gz`

[kpamnany]

CMake Error at lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt:20 (message):
  git clone https://github.com/intel/ittapi.git failed with No such file or
  directory, please clone https://github.com/intel/ittapi.git


-- Configuring incomplete, errors occurred!

@vchuravy had asked to check the value for WITH_ITTAPI. It's set to 0, as in the 1.9.1 tagged commit. Changing it to 1 did not help.

LLVM's CMakeCache.txt contained:

419 //No help, variable specified on the command line.
420 ITTAPI_SOURCE_DIR:UNINITIALIZED=/tmp/julia-1.9.1/deps/srccache/ittapi-0014aec56fea2f30c1374f40861e1bccdd53d0cb

There exists deps/srccache/ittapi-0014aec56fea2f30c1374f40861e1bccdd53d0cb.tar.gz, but it hasn't been extracted?

[kpamnany]

Cc: @vchuravy and @KristofferC

Need help with this please?

[vchuravy]

So this was meant to be fixed by #49022 and I see @nalimilan made a comment after merge https://github.com/JuliaLang/julia/pull/49022/files#r1162526538 that I didn't see.

[vchuravy]

See https://github.com/llvm/llvm-project/blob/c8447dd94422bb4740ccb181cd516236a0d19db6/llvm/lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt#L15

The cmake file seems to expect that the ${ITTAPI_SOURCE_DIR}‎ points to the parent folder that contains ittapi, whereas we point it to the actual source directory.

 make print-ITTAPI_SRC_DIR -C deps
make: Entering directory '/home/vchuravy/src/julia/deps'
ITTAPI_SRC_DIR=ittapi-0014aec56fea2f30c1374f40861e1bccdd53d0cb
make: Leaving directory '/home/vchuravy/src/julia/deps'

On top of that we don't follow the naming convention.

Possible fixes:

1. Patch LLVM
2. Create a symlink?

@nalimilan any preferences?

[kpamnany]

We can patch 1.9.1-full right after unpacking it. Will simply applying @nalimilan's suggestion work?

[vchuravy]

As @nalimilan pointed out that is not fully sufficient. You need to also do what I described #50127 (comment)

[kpamnany]

Successfully built Julia 1.9.1 from the full tarball (USE_BINARYBUILDER=0 and USE_GPL_LIBS=0)! 🎉

Now I simply need a little help to automate the second and third steps below so that we can un-tar the full tarball, use patch to make these changes, and then make.

Steps:

1. Patched deps/llvm.mk with @nalimilan's suggestion:

291c291
< extract-llvm: $(SRCCACHE)/$(ITTAPI_SRC_DIR)/source-extracted
---
> $(SRCCACHE)/$(LLVM_SRC_DIR)/source-extracted: $(SRCCACHE)/$(ITTAPI_SRC_DIR)/source-extracted

2. Added a symlink after extraction:

~/julia-1.9.1$ ln -s deps/srccache/ittapi-0014aec56fea2f30c1374f40861e1bccdd53d0cb/ittapi deps/srccache/ittapi-0014aec56fea2f30c1374f40861e1bccdd53d0cb

3. Patched deps/srccache/llvm-julia-14.0.6-3/llvm/lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt after extraction with:

22d21
< endif()
24,28c23,28
< execute_process(COMMAND ${GIT_EXECUTABLE} checkout ${ITTAPI_GIT_TAG}
<                 WORKING_DIRECTORY ${ITTAPI_SOURCE_DIR}/ittapi
<                 RESULT_VARIABLE GIT_CHECKOUT_RESULT)
< if(NOT GIT_CHECKOUT_RESULT EQUAL "0")
<     message(FATAL_ERROR "git checkout ${ITTAPI_GIT_TAG} failed with ${GIT_CHECKOUT_RESULT}, please checkout ${ITTAPI_GIT_TAG} at ${ITTAPI_SOURCE_DIR}/ittapi")
---
>     execute_process(COMMAND ${GIT_EXECUTABLE} checkout ${ITTAPI_GIT_TAG}
>                     WORKING_DIRECTORY ${ITTAPI_SOURCE_DIR}/ittapi
>                     RESULT_VARIABLE GIT_CHECKOUT_RESULT)
>     if(NOT GIT_CHECKOUT_RESULT EQUAL "0")
>         message(FATAL_ERROR "git checkout ${ITTAPI_GIT_TAG} failed with ${GIT_CHECKOUT_RESULT}, please checkout ${ITTAPI_GIT_TAG} at ${ITTAPI_SOURCE_DIR}/ittapi")
>     endif()

The IntelJITEvents CMakeLists.txt first tests for the presence of ${ITTAPI_SOURCE_DIR}/ittapi and if it is not present, uses git to clone it. The first two steps above fix this. It then unconditionally tries to check out the ${ITTAPI_GIT_TAG} branch, which it cannot since the extracted ittapi is not a git repository. To fix this, the third step above moves the git branch check out inside the if check for the git clone.

[nalimilan]

Thanks for finding the solution! It doesn't sound too hard to automate these steps. For step 2, can't you just call ln -sf from the $(LLVM_BUILDDIR_withtype)/build-configured target in deps/llvm.mk? For step 3, I think you can just make a PR against https://github.com/JuliaLang/llvm-project/, which we use instead of vanilla LLVM.

[kpamnany]

Thanks for the tip on where to add step 2. Trying that now.

I'll open a PR to JuliaLang/llvm-project, but that won't help the full tarball, right? I'll have to apply the patch above after unpacking the tarball. Should I do the patch application also in deps/llvm.mk in the same target (build-configured)?

[vchuravy]

For this I would prefer a patch in deps/llvm.mk like in the good old times.

[kpamnany]

Example please @vchuravy?

[kpamnany]

Here's what I ended up doing:

diff --git a/deps/llvm.mk b/deps/llvm.mk
index 93e4cd0b80..673f2f8ae7 100644
--- a/deps/llvm.mk
+++ b/deps/llvm.mk
@@ -230,6 +230,8 @@ $$(LLVM_BUILDDIR_withtype)/build-compiled: $$(SRCCACHE)/$$(LLVM_SRC_DIR)/$1.patc
 LLVM_PATCH_PREV := $$(SRCCACHE)/$$(LLVM_SRC_DIR)/$1.patch-applied
 endef

+$(eval $(call LLVM_PATCH,llvm-ittapi-cmake))
+
 ifeq ($(USE_SYSTEM_ZLIB), 0)
 $(LLVM_BUILDDIR_withtype)/build-configured: | $(build_prefix)/manifest/zlib
 endif
@@ -288,7 +290,7 @@ fastcheck-llvm: #none
 check-llvm: $(LLVM_BUILDDIR_withtype)/build-checked

 ifeq ($(USE_INTEL_JITEVENTS),1)
-extract-llvm: $(SRCCACHE)/$(ITTAPI_SRC_DIR)/source-extracted
+$(SRCCACHE)/$(LLVM_SRC_DIR)/source-extracted: $(SRCCACHE)/$(ITTAPI_SRC_DIR)/source-extracted
 endif

 #todo: LLVM make check target is broken on julia.mit.edu (and really slow elsewhere)
diff --git a/deps/patches/llvm-ittapi-cmake.patch b/deps/patches/llvm-ittapi-cmake.patch
new file mode 100644
index 0000000000..6746d21754
--- /dev/null
+++ b/deps/patches/llvm-ittapi-cmake.patch
@@ -0,0 +1,47 @@
+diff --git a/lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt b/lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt
+index 0c5017c359d6..92777133e9de 100644
+--- a/lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt
++++ b/lib/ExecutionEngine/IntelJITEvents/CMakeLists.txt
+@@ -12,23 +12,23 @@ if(NOT DEFINED ITTAPI_SOURCE_DIR)
+     set(ITTAPI_SOURCE_DIR ${PROJECT_BINARY_DIR})
+ endif()
+
+-if(NOT EXISTS ${ITTAPI_SOURCE_DIR}/ittapi)
+-    execute_process(COMMAND ${GIT_EXECUTABLE} clone ${ITTAPI_GIT_REPOSITORY}
+-                    WORKING_DIRECTORY ${ITTAPI_SOURCE_DIR}
++if(NOT EXISTS ${ITTAPI_SOURCE_DIR})
++    execute_process(COMMAND ${GIT_EXECUTABLE} clone ${ITTAPI_GIT_REPOSITORY} ${ITTAPI_SOURCE_DIR}
++                    WORKING_DIRECTORY ${ITTAPI_SOURCE_DIR}/..
+                     RESULT_VARIABLE GIT_CLONE_RESULT)
+     if(NOT GIT_CLONE_RESULT EQUAL "0")
+         message(FATAL_ERROR "git clone ${ITTAPI_GIT_REPOSITORY} failed with ${GIT_CLONE_RESULT}, please clone ${ITTAPI_GIT_REPOSITORY}")
+     endif()
+-endif()
+
+-execute_process(COMMAND ${GIT_EXECUTABLE} checkout ${ITTAPI_GIT_TAG}
+-                WORKING_DIRECTORY ${ITTAPI_SOURCE_DIR}/ittapi
+-                RESULT_VARIABLE GIT_CHECKOUT_RESULT)
+-if(NOT GIT_CHECKOUT_RESULT EQUAL "0")
+-    message(FATAL_ERROR "git checkout ${ITTAPI_GIT_TAG} failed with ${GIT_CHECKOUT_RESULT}, please checkout ${ITTAPI_GIT_TAG} at ${ITTAPI_SOURCE_DIR}/ittapi")
++    execute_process(COMMAND ${GIT_EXECUTABLE} checkout ${ITTAPI_GIT_TAG}
++                    WORKING_DIRECTORY ${ITTAPI_SOURCE_DIR}
++                    RESULT_VARIABLE GIT_CHECKOUT_RESULT)
++    if(NOT GIT_CHECKOUT_RESULT EQUAL "0")
++        message(FATAL_ERROR "git checkout ${ITTAPI_GIT_TAG} failed with ${GIT_CHECKOUT_RESULT}, please checkout ${ITTAPI_GIT_TAG} at ${ITTAPI_SOURCE_DIR}")
++    endif()
+ endif()
+
+-include_directories( ${ITTAPI_SOURCE_DIR}/ittapi/include/ )
++include_directories( ${ITTAPI_SOURCE_DIR}/include/ )
+
+ if( HAVE_LIBDL )
+     set(LLVM_INTEL_JIT_LIBS ${CMAKE_DL_LIBS})
+@@ -40,7 +40,7 @@ set(LLVM_INTEL_JIT_LIBS ${LLVM_PTHREAD_LIB} ${LLVM_INTEL_JIT_LIBS})
+ add_llvm_component_library(LLVMIntelJITEvents
+   IntelJITEventListener.cpp
+   jitprofiling.c
+-  ${ITTAPI_SOURCE_DIR}/ittapi/src/ittnotify/ittnotify_static.c
++  ${ITTAPI_SOURCE_DIR}/src/ittnotify/ittnotify_static.c
+
+   LINK_LIBS ${LLVM_INTEL_JIT_LIBS}
+

When applied to the unzipped -full tarball, this lets LLVM build correctly.

[kpamnany]

Unfortunately, I then get:

<snip>
Future  ───────────  0.005029 seconds
InteractiveUtils  ─  0.493602 seconds
error during bootstrap:
LoadError("sysimg.jl", 19, LoadError("/build/source/usr/share/julia/stdlib/v1.9/LibGit2/src/LibGit2.jl", 3, LoadError("/build/source/usr/share/julia/stdlib/v1.9/LibGit2/src/utils.jl", 44, ErrorException("could not load library \"libgit2\"\nlibgit2.so: cannot open shared object file: No such file or directory"))))
ijl_errorf at /build/source/src/rtutils.c:77
ijl_load_dynamic_library at /build/source/src/dlload.c:369
jl_get_library_ at /build/source/src/runtime_ccall.cpp:48 [inlined]
jl_get_library_ at /build/source/src/runtime_ccall.cpp:30
ijl_load_and_lookup at /build/source/src/runtime_ccall.cpp:61
unknown function (ip: 0x7fffdd8606bf)
macro expansion at /build/source/usr/share/julia/stdlib/v1.9/LibGit2/src/error.jl:109 [inlined]
version at /build/source/usr/share/julia/stdlib/v1.9/LibGit2/src/utils.jl:40
<snip>

Suggestions welcome.