libgit issues with Julia 1.6.5 h989b2f6_0

[mkitti]

There seem to be potential libgit2 issues with Julia 1.6.5 h989b2f6_0

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.5 (2021-12-19)
 _/ |\__'_|_|_|\__'_|  |  conda-forge-julia release
|__/                   |

(@julia171_test) pkg> activate --temp
  Activating new environment at `/tmp/jl_wZ4SZa/Project.toml`

(jl_wZ4SZa) pkg> add https://github.com/mkitti/libcxxwrap_julia_jll_mock.git#cf
    Updating git-repo `https://github.com/mkitti/libcxxwrap_julia_jll_mock.git`
ERROR: failed to fetch from https://github.com/mkitti/libcxxwrap_julia_jll_mock.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)

(jl_wZ4SZa) pkg> add https://github.com/mkitti/libcxxwrap_julia_jll_mock.git
    Updating git-repo `https://github.com/mkitti/libcxxwrap_julia_jll_mock.git`
ERROR: failed to fetch from https://github.com/mkitti/libcxxwrap_julia_jll_mock.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)


$ conda list
# packages in environment at /home/mkitti/anaconda3/envs/julia171_test:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
arpack                    3.7.0                hdefa2d7_2    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
curl                      7.80.0               h494985f_1    conda-forge
expat                     2.4.2                h9c3ff4c_0    conda-forge
gettext                   0.19.8.1          h73d1719_1008    conda-forge
git                       2.34.1          pl5321hf874766_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
julia                     1.6.5                h989b2f6_0    conda-forge
krb5                      1.19.2               h48eae69_3    conda-forge
libblas                   3.9.0           12_linux64_openblas    conda-forge
libcblas                  3.9.0           12_linux64_openblas    conda-forge
libcurl                   7.80.0               h494985f_1    conda-forge
libcxxwrap-julia          0.8.3                h4bd325d_1    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgfortran-ng            11.2.0              h69a702a_11    conda-forge
libgfortran5              11.2.0              h5c6108e_11    conda-forge
libgit2                   1.3.0                haabb1ae_1    conda-forge
libgomp                   11.2.0              h1d223b6_11    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0           12_linux64_openblas    conda-forge
libnghttp2                1.43.0               ha19adfc_1    conda-forge
libopenblas               0.3.18          pthreads_h8fe5266_0    conda-forge
libopenblas-ilp64         0.3.18          pthreads_h0aa5bea_0    conda-forge
libssh2                   1.10.0               ha35d2d1_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_11    conda-forge
libunwind                 1.5.0                h9c3ff4c_1    conda-forge
libutf8proc               2.7.0                h7f98852_0    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
mbedtls                   3.1.0                h9c3ff4c_0    conda-forge
metis                     5.1.0             h58526e2_1006    conda-forge
mpfr                      4.1.0                h9202a9a_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
openblas-ilp64            0.3.18          pthreads_h3d04fff_0    conda-forge
openlibm                  0.7.0                h516909a_0    conda-forge
openssl                   3.0.0                h7f98852_2    conda-forge
p7zip                     16.02             he1b5a44_1000    conda-forge
pcre2                     10.37                h032f7d1_0    conda-forge
perl                      5.32.1          1_h7f98852_perl5    conda-forge
suitesparse               5.10.1               h9e50725_1    conda-forge
tbb                       2021.5.0             h4bd325d_0    conda-forge
tk                        8.6.11               h27826a3_1    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge

[mkitti]

This seems to work fine:

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.2 (2021-07-14)
 _/ |\__'_|_|_|\__'_|  |  conda-forge-julia release
|__/                   |

(@v1.6) pkg> activate --temp
  Activating new environment at `/tmp/jl_VdEGFV/Project.toml`

(jl_VdEGFV) pkg> add https://github.com/mkitti/libcxxwrap_julia_jll_mock.git#cf
    Updating git-repo `https://github.com/mkitti/libcxxwrap_julia_jll_mock.git`
    Updating registry at `~/.julia/registries/General`
   Resolving package versions...
    Updating `/tmp/jl_VdEGFV/Project.toml`
  [3eaa8342] + libcxxwrap_julia_jll v0.8.8+1 `https://github.com/mkitti/libcxxwrap_julia_jll_mock.git#cf`
    Updating `/tmp/jl_VdEGFV/Manifest.toml`
  [3eaa8342] + libcxxwrap_julia_jll v0.8.8+1 `https://github.com/mkitti/libcxxwrap_julia_jll_mock.git#cf`

$ conda list
# packages in environment at /home/mkitti/anaconda3/envs/julia162_test:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
arpack                    3.7.0                hdefa2d7_2    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.17.2               h7f98852_0    conda-forge
ca-certificates           2021.5.30            ha878542_0    conda-forge
curl                      7.78.0               hea6ffbf_0    conda-forge
expat                     2.4.1                h9c3ff4c_0    conda-forge
gettext                   0.19.8.1          h73d1719_1006    conda-forge
git                       2.33.0          pl5321hc30692c_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
julia                     1.6.2                h96f571d_0    conda-forge
krb5                      1.19.2               hcc1bbae_0    conda-forge
libblas                   3.9.0           11_linux64_openblas    conda-forge
libcblas                  3.9.0           11_linux64_openblas    conda-forge
libcurl                   7.78.0               h2574ce0_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h9c3ff4c_1    conda-forge
libgcc-ng                 11.1.0               hc902ee8_8    conda-forge
libgfortran-ng            11.1.0               h69a702a_8    conda-forge
libgfortran5              11.1.0               h6c583b3_8    conda-forge
libgit2                   1.1.1                hee63804_1    conda-forge
libgomp                   11.1.0               hc902ee8_8    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0           11_linux64_openblas    conda-forge
libnghttp2                1.43.0               h812cca2_0    conda-forge
libopenblas               0.3.17          pthreads_h8fe5266_1    conda-forge
libopenblas-ilp64         0.3.17          pthreads_h0aa5bea_1    conda-forge
libssh2                   1.10.0               ha56f1ee_0    conda-forge
libstdcxx-ng              11.1.0               h56837e0_8    conda-forge
libunwind                 1.5.0                h9c3ff4c_1    conda-forge
libutf8proc               2.6.1                h7f98852_0    conda-forge
metis                     5.1.0             h58526e2_1006    conda-forge
mpfr                      4.1.0                h9202a9a_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
openblas-ilp64            0.3.17          pthreads_h3d04fff_1    conda-forge
openlibm                  0.7.0                h516909a_0    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
p7zip                     16.02             he1b5a44_1000    conda-forge
pcre2                     10.37                h032f7d1_0    conda-forge
perl                      5.32.1          0_h7f98852_perl5    conda-forge
suitesparse               5.4.0                h717dc36_0    conda-forge
tbb                       2020.2               h4bd325d_4    conda-forge
tk                        8.6.11               h27826a3_1    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge

[mkitti]

I see. 1.6 branch probably needs 233c339

Fix in #159 ?

[ngam]

Ugh... I thought the libgit patch wasn't needed for the 1.6.x versions... We may want to actually solve this more properly later; I think we could work on constraining the exact version they're relying on upstream. I remember looking into it (you either apply the patch) or constrain libgit like libunwind, etc.

[mkitti]

Upstream is pinned at libgit2 version 1.1.0.
https://github.com/JuliaLang/julia/blob/v1.6.5/deps/libgit2.version

[mkitti]

See also

JuliaLang/julia#42209 (libgit2 version 1.2)
JuliaLang/julia#43250 (libgit2 version 1.3)

[ngam]

Upstream is pinned at libgit2 version 1.1.0

so the patch in 233c339 won't make a difference then.

+    @static if LibGit2.VERSION >= v"1.2.0"
+        remote_ready::Ptr{Cvoid}       = C_NULL
+    end

[ngam]

nvm sorry, it should take care of it, I think

[mkitti]

1.7.1 is pinned at the same version, and the patch does seem to work.

[ngam]

1.7.1 is pinned at the same version, and the patch does seem to work.

Yep, sorry I wrote that without thinking! Will hide the comment :)

[MilesCranmer]

Sorry to ping this old thread - I expanded the PySR unit tests to include conda-forge on macOS, and am seeing this issue here: https://github.com/MilesCranmer/PySR/actions/runs/3060750059/jobs/4939640457#step:5:155. Any idea why this is occuring?

[mkitti]

The fix is to unset the environment variable JULIA_SSL_CA_ROOTS_PATH on macOS.

julia-feedstock/recipe/scripts/activate.sh

Lines 20 to 22 in a203b65

	if [[ $OSTYPE != 'darwin'* ]]; then
	export JULIA_SSL_CA_ROOTS_PATH=$CONDA_PREFIX/ssl/cacert.pem
	fi

I think you need to unpin Julia from 1.7.1.

[MilesCranmer]

Awesome, thanks! Will unpin pysr now. Is it possible to backport it, just to generalize the solution to other versions?

[mkitti]

The main issue is that Julia's mbedtls doesn't understand conda-forge's openssl certificate format.