"error initializing LibGit2 module" when CA certificates not found

[colbec]

Using openSUSE Leap 42.1:
24 hours ago a git pull followed by a make && make testall ran fine. This morning I had an error in make:

...
patching file tests/online/badssl.c
CMake Error: The current CMakeCache.txt directory /home/colin/Downloads/julia6/deps/scratch/libgit2-211e117a0590583a720c53172406f34186c543bd/CMakeCache.txt is different than the directory /home/colin/Downloads/julia/deps/scratch/libgit2-211e117a0590583a720c53172406f34186c543bd where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
CMake Error: The source "/home/colin/Downloads/julia6/deps/srccache/libgit2-211e117a0590583a720c53172406f34186c543bd/CMakeLists.txt" does not match the source "/home/colin/Downloads/julia/deps/srccache/libgit2-211e117a0590583a720c53172406f34186c543bd/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
/home/colin/Downloads/julia6/deps/libgit2.mk:66: recipe for target 'scratch/libgit2-211e117a0590583a720c53172406f34186c543bd/build-configured' failed
make[1]: *** [scratch/libgit2-211e117a0590583a720c53172406f34186c543bd/build-configured] Error 1
Makefile:81: recipe for target 'julia-deps' failed
make: *** [julia-deps] Error 2

I thought I had fixed this with make -C deps distclean-libgit2 distclean-mbedtls distclean-libssh2 which allowed make to run to the end. However now Julia 0.6 fails to launch with:

> julia6
fatal: error thrown and no exception handler available.
Base.InitError(mod=:LibGit2, error=ErrorException("error initializing LibGit2 module"))
rec_backtrace at /home/colin/Downloads/julia6/src/stackwalk.c:84
record_backtrace at /home/colin/Downloads/julia6/src/task.c:238
jl_throw at /home/colin/Downloads/julia6/src/task.c:560
__init__ at ./libgit2/libgit2.jl:539
unknown function (ip: 0x7f60d03e3e08)
jl_call_method_internal at /home/colin/Downloads/julia6/src/julia_internal.h:218 [inlined]
jl_apply_generic at /home/colin/Downloads/julia6/src/gf.c:1852
jl_apply at /home/colin/Downloads/julia6/src/julia.h:1376 [inlined]
jl_module_run_initializer at /home/colin/Downloads/julia6/src/toplevel.c:83
_julia_init at /home/colin/Downloads/julia6/src/init.c:702
julia_init at /home/colin/Downloads/julia6/src/task.c:289
unknown function (ip: 0x4014cd)
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x40151f)

Prebuilt Julia 0.5 works fine.

[tkelman]

What does ldd usr/lib/libgit2.so say?

[tkelman]

If you see anything in there about libssl or libcrypto (but not of the mbed variety), try make cleanall && make

[colbec]

One curiosity is that if I do a find ./ -name libgit2.so from julia main directory I find eight different files matched, some clearly symlinks, but others may be different residual versions causing confusion.

> ldd usr/lib/libgit2.so
    linux-vdso.so.1 (0x00007ffeced63000)
    libcurl.so.4 => /home/colin/Downloads/julia6/usr/lib/libcurl.so.4 (0x00007f5926bf8000)
    libz.so.1 => /lib64/libz.so.1 (0x00007f59269b2000)
    libmbedtls.so.10 => /home/colin/Downloads/julia6/usr/lib/libmbedtls.so.10 (0x00007f592678b000)
    libmbedx509.so.0 => /home/colin/Downloads/julia6/usr/lib/libmbedx509.so.0 (0x00007f5926577000)
    libmbedcrypto.so.0 => /home/colin/Downloads/julia6/usr/lib/libmbedcrypto.so.0 (0x00007f5926323000)
    libssh2.so.1 => /home/colin/Downloads/julia6/usr/lib/libssh2.so.1 (0x00007f59260f0000)
    librt.so.1 => /lib64/librt.so.1 (0x00007f5925ee8000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5925cca000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f5925922000)
    /lib64/ld-linux-x86-64.so.2 (0x00005572745ee000)

Then:
make cleanall is successful, but make finishes with:

...
configure: creating ./config.status
config.status: creating libuv.pc
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands
make[2]: *** No rule to make target '/home/colin/Downloads/julia/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/src/fs-poll.c', needed by 'src/libuv_la-fs-poll.lo'.  Stop.
/home/colin/Downloads/julia6/deps/libuv.mk:37: recipe for target 'scratch/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/build-compiled' failed
make[1]: *** [scratch/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/build-compiled] Error 2
Makefile:81: recipe for target 'julia-deps' failed
make: *** [julia-deps] Error 2

[tkelman]

I'm not sure what to make of the libuv failure. Does it persist after make -C deps distclean-libuv ?

[tkelman]

Can you also check ldd on some of the other libraries, libcurl, libssh2, and the libmbed* ? Is anything obviously missing?

[colbec]

After make -C deps distclean-libuv make still failing.

> make
 cd /home/colin/Downloads/julia/deps/srccache/libunwind-1.1-julia2 && /bin/sh /home/colin/Downloads/julia/deps/srccache/libunwind-1.1-julia2/config/missing automake-1.15 --gnu Makefile
/bin/sh: line 10: cd: /home/colin/Downloads/julia/deps/srccache/libunwind-1.1-julia2: No such file or directory
Makefile:477: recipe for target '/home/colin/Downloads/julia/deps/srccache/libunwind-1.1-julia2/Makefile.in' failed
make[2]: *** [/home/colin/Downloads/julia/deps/srccache/libunwind-1.1-julia2/Makefile.in] Error 1
/home/colin/Downloads/julia6/deps/unwind.mk:31: recipe for target '/home/colin/Downloads/julia6/usr-staging/libunwind-1.1-julia2.tgz' failed
make[1]: *** [/home/colin/Downloads/julia6/usr-staging/libunwind-1.1-julia2.tgz] Error 2
Makefile:81: recipe for target 'julia-deps' failed
make: *** [julia-deps] Error 2

[tkelman]

Something is very wrong, but I'm not sure if it's in Julia's makefiles or local to your particular clone. Does a build from scratch work correctly (maybe in a new clone to at least keep this build's copies of llvm and openblas around if they're still working)?

[colbec]

The directory usr/lib does not contain a libcurl,so or libssh2.so or libmbed*.so. Should they be there? Doing a find for libcurl.so finds them in .deps/build and scratch.

I will wipe the cache and try again.

[colbec]

@tkelman Just finished a re-clone (everything deleted and recreated from scratch) of latest v 0.6 dev, and make is successful but again, will not launch as above.

[vchuravy]

I just hit this on power:

    JULIA test/all
fatal: error thrown and no exception handler available.
Base.InitError(mod=:LibGit2, error=ErrorException("error initializing LibGit2 module"))
rec_backtrace at /home/valentin/juliatest/src/stackwalk.c:84
record_backtrace at /home/valentin/juliatest/src/task.c:238
jl_throw at /home/valentin/juliatest/src/task.c:560
__init__ at ./libgit2/libgit2.jl:539
unknown function (ip: 0x3fffa68cf1bf)
jl_call_method_internal at /home/valentin/juliatest/src/julia_internal.h:239 [inlined]
jl_apply_generic at /home/valentin/juliatest/src/gf.c:1852
jl_apply at /home/valentin/juliatest/src/julia.h:1376 [inlined]
jl_module_run_initializer at /home/valentin/juliatest/src/toplevel.c:83
_julia_init at /home/valentin/juliatest/src/init.c:702
julia_init at /home/valentin/juliatest/src/task.c:289
main at /home/valentin/juliatest/ui/repl.c:247
unknown function (ip: 0x3fffb226457f)
__libc_start_main at /lib64/power8/libc.so.6 (unknown line)
make: *** [all] Error 1
make: Leaving directory `/home/valentin/juliatest/test'

[maleadt]

Same here, Arch Linux x86-64, both libgit2 and libssh2 are properly linked to mbedtls. Seems like git_mbedtls_stream_global_init is returning -1.

[maleadt]

It isn't finding the CA certificates, after trying:

• ENV[X509_CERT_FILE_EVP="SSL_CERT_FILE"]
• ENV[X509_CERT_DIR_EVP="SSL_CERT_DIR"]
• "/usr/lib/ssl/cert.pem"
• "/usr/lib/ssl/certs"

In other words, this is failing.

EDIT: on my Debian systems, "/usr/lib/ssl/certs" symlinks to "/etc/ssl/certs", but on Arch this is not the case. Maybe we should add "/etc/ssl/certs" to the search path? As a workaround, exporting SSL_CERT_DIR makes Julia work again.