Compile from source without pre-built deps failed

[inkydragon]

error list

• blastrampoline
  • source build error: fixed by [deps/blastrampoline]: pass cross compile flags to make #45909
  • No rule to make target: win: fix "No rule to make target" JuliaLinearAlgebra/libblastrampoline#107
  • libblastrampoline_jll load error
• zlib source build error: fixed by [deps/zlib]: fix source build #45912
• llvm
  • source build error
  • link error: fixed by [deps/llvm] don't use hardcode LLVM_SHARED_LIB_NAME #45908
  • install error
• libgit2 source build error
• curl
  • curl source build error: fixed by libssh2 patch [deps/libssh2]: fix import library name #45914
  • libcurl load error: fixed by [deps/curl]: sync build flags with Yggdrasil/LibCURL #45933
• p7zip source build error

Some patches may need to be backported to v1.7, v1.8

env

• Win 10 21H2
• CYGWIN_NT-10.0 wos-PC 3.3.4(0.341/5/3) 2022-01-31 19:35 x86_64 Cygwin
• cc, gcc, g++, x86_64-w64-mingw32-gcc, x86_64-w64-mingw32-gfortran 11.2.0
• GNU Make 4.3
• cmake version 3.20.0
• Python 3.9.10

Make.user

XC_HOST = x86_64-w64-mingw32
USE_BINARYBUILDER=0

compile steps:

make O=julia-win64 configure
echo 'XC_HOST = x86_64-w64-mingw32' > julia-win64/Make.user
echo 'USE_BINARYBUILDER=0' >> julia-win64/Make.user
echo 'ifeq ($(BUILDROOT),$(JULIAHOME))
        $(error "in-tree build disabled")
      endif' >> Make.user
make -C julia-win64

julia source verison master

[inkydragon]

zlib

c++ - Why does compiling zlib with MSVC compilers work fine and yet gives a syntax error with Cygwin? - Stack Overflow

https://github.com/madler/zlib/blob/21767c654d31d2dccdde4330529775c6c5fd5389/CMakeLists.txt#L170-L174
It looks like cmake doesn't think I'm using MINGW (MINGW==false).

Maybe related: CMake 3.22: MINGW is false with mingw toolchains on linux (#22948) · Issues · CMake / CMake · GitLab

patch

diff --git a/deps/zlib.mk b/deps/zlib.mk
index d43f829c13..a5257612d8 100644
--- a/deps/zlib.mk
+++ b/deps/zlib.mk
@@ -6,7 +6,7 @@ $(eval $(call git-external,zlib,ZLIB,,,$(SRCCACHE)))

 $(BUILDDIR)/$(ZLIB_SRC_DIR)/build-configured: $(SRCCACHE)/$(ZLIB_SRC_DIR)/source-extracted
        mkdir -p $(dir $@)
-       cd $(dir $@) && $(CMAKE) -DCMAKE_INSTALL_PREFIX=$(abspath $(build_prefix)) -DCMAKE_BUILD_TYPE=Release -DUNIX=true $(dir $<)
+       cd $(dir $@) && $(CMAKE) $(CMAKE_COMMON) -DCMAKE_BUILD_TYPE=Release -DUNIX=true $(dir $<)
        echo 1 > $@

 $(BUILDDIR)/$(ZLIB_SRC_DIR)/build-compiled: $(BUILDDIR)/$(ZLIB_SRC_DIR)/build-configured

pr: #45912

[inkydragon]

llvm build error

long error log

error

Consolidate compiler generated dependencies of target LLVMSupport
make[8]: Leaving directory '/cygdrive/v/julia/julia-win64/deps/llvm-julia-13.0.1-0/build_Release/NATIVE'
make  -f lib/Support/CMakeFiles/LLVMSupport.dir/build.make lib/Support/CMakeFiles/LLVMSupport.dir/build
make[8]: Entering directory '/cygdrive/v/julia/julia-win64/deps/llvm-julia-13.0.1-0/build_Release/NATIVE'
[  0%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/ThreadLocal.cpp.o
cd /cygdrive/v/julia/julia-win64/deps/llvm-julia-13.0.1-0/build_Release/NATIVE/lib/Support && /usr/bin/g++ -DGTEST_HAS_RTTI=0 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/cygdrive/v/julia/julia-win64/deps/llvm-julia-13.0.1-0/build_Release/NATIVE/lib/Support -I/cygdrive/v/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Support -I/cygdrive/v/julia/julia-win64/deps/llvm-julia-13.0.1-0/build_Release/NATIVE/include -I/cygdrive/v/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/include -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wmisleading-indentation -O3 -DNDEBUG -std=gnu++14  -fno-exceptions -fno-rtti -MD -MT lib/Support/CMakeFiles/LLVMSupport.dir/ThreadLocal.cpp.o -MF CMakeFiles/LLVMSupport.dir/ThreadLocal.cpp.o.d -o CMakeFiles/LLVMSupport.dir/ThreadLocal.cpp.o -c /cygdrive/v/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Support/ThreadLocal.cpp
In file included from /cygdrive/v/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Support/ThreadLocal.cpp:42:
/cygdrive/v/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Support/Unix/ThreadLocal.inc: In member function 'void llvm::sys::ThreadLocalImpl::setInstance(const void*)':
/cygdrive/v/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Support/Unix/ThreadLocal.inc:66:57: error: incompatible types in assignment of 'void*' to 'char [8]'
   66 | void ThreadLocalImpl::setInstance(const void* d) { data = const_cast<void*>(d);}
      |                                                    ~~~~~^~~~~~~~~~~~~~~~~~~~~~
make[8]: *** [lib/Support/CMakeFiles/LLVMSupport.dir/build.make:1952: lib/Support/CMakeFiles/LLVMSupport.dir/ThreadLocal.cpp.o] Error 1

make call /usr/bin/g++, It is probably that the compile flag was not passed correctly.

Or we may update LLVM to julia-13.0.1-1.

solution

fix patch
Maybe this is a bug, and need to backport a patch?

JuliaLang/llvm-project@d351f54

link error: cannot find lib

julia/deps/llvm.mk

Lines 200 to 204 in a9e3cc7

	# change the SONAME of Julia's private LLVM
	# i.e. libLLVM-6.0jl.so
	# see #32462
	LLVM_CMAKE += -DLLVM_VERSION_SUFFIX:STRING="jl"
	LLVM_CMAKE += -DLLVM_SHLIB_SYMBOL_VERSION:STRING="JL_LLVM_$(LLVM_VER_SHORT)"

solution

libLLVM.dll now called libLLVM-13jl.dll

fix patch

diff --git a/deps/llvm.mk b/deps/llvm.mk
index e0512137da..ecc7756e03 100644
--- a/deps/llvm.mk
+++ b/deps/llvm.mk
@@ -258,7 +258,7 @@ LLVM_INSTALL = \
     cp -r $$(SRCCACHE)/$$(LLVM_SRC_DIR)/llvm/utils/lit $2$$(build_depsbindir)/ && \
     $$(CMAKE) -DCMAKE_INSTALL_PREFIX="$2$$(build_prefix)" -P cmake_install.cmake
 ifeq ($(OS), WINNT)
-LLVM_INSTALL += && cp $2$$(build_shlibdir)/libLLVM.dll $2$$(build_depsbindir)
+LLVM_INSTALL += && cp $2$$(build_shlibdir)/libLLVM-*.dll $2$$(build_depsbindir)/libLLVM.dll
 endif
 ifeq ($(OS),Darwin)
 # https://github.com/JuliaLang/julia/issues/29981

Maybe macOS's command need a patch too?

julia/deps/llvm.mk

Lines 260 to 266 in a9e3cc7

	ifeq ($(OS), WINNT)
	LLVM_INSTALL += && cp $2$$(build_shlibdir)/libLLVM.dll $2$$(build_depsbindir)
	endif
	ifeq ($(OS),Darwin)
	# https://github.com/JuliaLang/julia/issues/29981
	LLVM_INSTALL += && ln -s libLLVM.dylib $2$$(build_shlibdir)/libLLVM-$$(LLVM_VER_SHORT).dylib
	endif

Intermittent build errors in cygwin

bash - cygwin cp can't copy file "abc" because destination dir. contains file "abc.exe" - Stack Overflow

julia/deps/llvm.mk

Line 258 in fa2f304

cp -r $$(SRCCACHE)/$$(LLVM_SRC_DIR)/llvm/utils/lit $2$$(build_depsbindir)/ && \

cygwin sometimes fails when copying test files which located in the llvm/utils/lit folder.

solution

• workaround: A possible solution is to force cp to always return true:

-    cp -r $$(SRCCACHE)/$$(LLVM_SRC_DIR)/llvm/utils/lit $2$$(build_depsbindir)/ && \
+    cp -r $$(SRCCACHE)/$$(LLVM_SRC_DIR)/llvm/utils/lit $2$$(build_depsbindir)/ || true && \

[inkydragon]

libgit2 <- libssh2

When building libgit2, sometimes libssh2 is not found

solution 1 - use mbedTLS for libssh2

I'm not sure if this patch is necessary or not. More tests needed.

patches

diff --git a/deps/libssh2.mk b/deps/libssh2.mk
index e27a57a407..8a17c814dd 100644
--- a/deps/libssh2.mk
+++ b/deps/libssh2.mk
@@ -11,11 +11,9 @@ endif
 LIBSSH2_OPTS := $(CMAKE_COMMON) -DBUILD_SHARED_LIBS=ON -DBUILD_EXAMPLES=OFF \
                -DCMAKE_BUILD_TYPE=Release
 
-ifeq ($(OS),WINNT)
-LIBSSH2_OPTS += -DCRYPTO_BACKEND=WinCNG -DENABLE_ZLIB_COMPRESSION=OFF        
 ifeq ($(BUILD_OS),WINNT)
+LIBSSH2_OPTS += -DCRYPTO_BACKEND=WinCNG -DENABLE_ZLIB_COMPRESSION=OFF        
 LIBSSH2_OPTS += -G"MSYS Makefiles"
-endif
 else
 LIBSSH2_OPTS += -DCRYPTO_BACKEND=mbedTLS -DENABLE_ZLIB_COMPRESSION=OFF       
 endif
diff --git a/deps/mbedtls.mk b/deps/mbedtls.mk
index 12788e1c03..5aefa98232 100644
--- a/deps/mbedtls.mk
+++ b/deps/mbedtls.mk
@@ -47,7 +47,7 @@ ifeq ($(OS),$(BUILD_OS))
 endif
        echo 1 > $@
 
-ifeq ($(OS),WINNT)
+ifeq ($(BUILD_OS),WINNT)
 define MBEDTLS_INSTALL
        mkdir -p $2/$$(build_shlibdir)
        cp $1/library/libmbedcrypto.$$(SHLIB_EXT) $2/$$(build_shlibdir)

solution 2 - use WinCNG for libssh2

and Update libgit2's build script: https://github.com/JuliaPackaging/Yggdrasil/blob/f73b0dc8ce7f6927c49d9616487092b52aff5a28/L/LibGit2/build_tarballs.jl#L36-L37

cmake cannot find bcrypt...

-LIBGIT2_OPTS += -DCMAKE_FIND_ROOT_PATH=/usr/$(XC_HOST) -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY
+LIBGIT2_OPTS += -DCMAKE_FIND_ROOT_PATH=/usr/$(XC_HOST)/sys-root/mingw/ -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY

[inkydragon]

curl <- libssh2

error msg

when configuring curl

checking for in_addr_t equivalent... unknown
configure: error: Cannot find a type to use in place of in_addr_t
make: *** [/cygdrive/v/julia/deps/curl.mk:49：curl-7.83.1/build-configured] 错误 1
make: 离开目录“/cygdrive/v/julia/julia-win64/deps”

Related issue: AUR (en) - mingw-w64-libssh2

Solution

• workaround: copy or rename liblibssh2.dll.a --> libssh2.dll.a

• upstream patch: Fix DLL import library name libssh2/libssh2#711
  fixed by: [deps/libssh2]: fix import library name #45914

[inkydragon]

p7zip

error msg

On Windows, cross compile with mingw in cygwin will give errors:

$ make CC="x86_64-w64-mingw32-gcc" 7za
mkdir -p bin
make -C CPP/7zip/Bundles/Alone all
make[1]: 进入目录“/cygdrive/v/tmp/p7zip-17.04/CPP/7zip/Bundles/Alone”
x86_64-w64-mingw32-gcc -c -std=gnu11 -I. -I../../../../C -I../../../../CPP/myWindows -I../../../../CPP/include_windows -I../../../../CPP -O2 -s -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DNDEBUG -D_REENTRANT -DENV_UNIX -D_7ZIP_LARGE_PAGES  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_REENTRANT -DENV_UNIX -DBREAK_HANDLER -DUNICODE -D_UNICODE -DUNIX_USE_WIN_FILE -DZSTD_MULTITHREAD -DNO_XXHASH -DFL2_7ZIP_BUILD   ../../../../C/Alloc.c
In file included from ../../../../C/Alloc.c:7:
../../../../CPP/include_windows/windows.h:81:9: 错误：未知的类型名‘WCHAR’
   81 | typedef WCHAR *LPWSTR;
      |         ^~~~~
../../../../CPP/include_windows/windows.h:82:9: 错误：未知的类型名‘CHAR’

Solution

• workaround: 7z is just an external tool, so I'm skipping the cross-compilation for now.

Maybe p7zip doesn't support compiling on native windows.
But why MSYS2 can compile it successfully? Package: p7zip - MSYS2 Packages

It's incredibly frustrating to build p7zip on mingw, so instead we just redistribute 7z

https://github.com/JuliaPackaging/Yggdrasil/blob/b2e0c2c5851b71230fb7170f74d773393ce37f80/P/p7zip/build_tarballs.jl#L21

We should learn from Yggdrasil's build scripts.

[inkydragon]

LibCURL.jl <- libcurl

During the build of sysimg, the shared library libcurl cannot be loaded when processing the standard library LibCURL.

LibCURL  ──────────  0.330104 seconds
error during bootstrap:
LoadError("sysimg.jl", 14, LoadError("V:\\julia\\julia-win64\\usr\\share\\julia\\stdlib\\v1.9\\Downloads\\src\\Downloads.jl", 1, LoadError("V:\\julia\\julia-win64\\usr\\share\\julia\\stdlib\\v1.9\\Downloads\\src\\Curl\\Curl.jl", 1, ErrorException("could not load library \"libcurl-4.dll\"\nThe specified module could not be found. "))))
ijl_errorf at /cygdrive/v/julia/src\rtutils.c:77

solution

• workaround: copy or rename: libcurl.dll -> libcurl-4.dll

cd usr/bin
cp libcurl.dll libcurl-4.dll

• patch: add configure flag --enable-versioned-symbols
  ref: https://github.com/JuliaPackaging/Yggdrasil/blob/b2e0c2c5851b71230fb7170f74d773393ce37f80/L/LibCURL/common.jl#L39

And we want versioned dll on win.

julia/deps/tools/common.mk

Lines 62 to 64 in fa2f304

	ifeq ($(OS), WINNT)
	LIBTOOL_CCLD := CCLD="$(CC) -no-undefined -avoid-version"
	endif

[inkydragon]

Finally, julia can be compiled successfully.

$ ./julia.bat
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-DEV.759 (2022-06-12)
 _/ |\__'_|_|_|\__'_|  |  offline-build/f97f25ea34* (fork: 4 commits, 1 day)
|__/                   |

julia> versioninfo()
Julia Version 1.9.0-DEV.759
Commit f97f25ea34* (2022-06-12 15:49 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 6 × Intel(R) Core(TM) i5-9400F CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 6 virtual cores

[metab0t]

Sadly many deps are not configured correctly to invoke correct cross compiler.

[ViralBShah]

@inkydragon Will you be making a PR with these?

[inkydragon]

@ViralBShah Yes, I will. Maybe I'll open a draft pr later.

At the moment, I haven't found the right patch for some problems and need to manually deal with them.

• p7zip

7z is just an external tool, so I'm skipping the cross-compilation for now.

Originally posted by @inkydragon in #45645 (comment)

Some issues may require upstream patches:

• LLVM

Maybe this is a bug, and need to backport a patch?

JuliaLang/llvm-project@d351f54

Originally posted by @inkydragon in #45645 (comment)

• BLT: Add support for DESTDIR JuliaLinearAlgebra/libblastrampoline#76

[metab0t]

@inkydragon
libssh2 name issue can be fixed by libssh2/libssh2#711