libstdc++.so.6: version `GLIBCXX_3.4.30' not found

[torrance]

I'm encountering an issue where:

1. A Conda package is built against libc 3.4.30 (in this case, python-casacore).
2. Conda ships with libc 3.4.30.
3. Running ldd libcasa_casa.so shows that casa is correctly linked to the Conda version of libc.
4. If I enter the Julia-managed conda (~/.julia/3/bin/python) and import casacore.tables it works fine.

BUT, when I pyimport("casacore.tables") I get the following error:

<class 'ImportError'>
ImportError("/home/torrance/julia-1.7.3/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/torrance/.julia/conda/3/lib/python3.9/site-packages/casacore/tables/../../../../libcasa_tables.so.7)")

It looks like to me that when run in the context of Julia, the pyimport statement wants to preferentially link against the Julia supplied libc, and not the Conda supplied libc. Moreover, the Julia supplied libc is at version 3.4.29.

Is this choice to link against the Julia libc a deliberate choice? If so, how to we mitigate this kind of issue happening each time Conda and Julia libc versions aren't aligned?

[joegilkes]

I've been having the same issue, my workaround for now has been to set the LD_PRELOAD environment variable whenever I start a Julia REPL/script, ie.

LD_PRELOAD=/path/to/my_conda_env/lib/libstdc++.so.6 julia

This is a bit inconvenient since I have to always run Julia through a separate bash script whenever I want to start it for this project, and I have to do the same thing if I want to run a Julia Jupyter notebook. The correct behaviour could be set as default within a project if there was a way to save project-specific permanent environment variables (e.g. running julia --project=xyz would automatically have the correct LD_PRELOAD set for that project), but I'm not aware of a way to do this at the moment.

[Naikless]

Same issue for me with some scipy.optimize modules imported via PyCall from the Julia-managed conda environment.

While the above solution of preloading the more recent library works as a local fix, it is no solution if these imports are part of a published package. You can't expect every user to check for this. Since conda should already provide a consistent environment in terms of binary libs, the choice to link conda packages to Julia binaries does not seem helpful to me and I would also like to know if there are technical reasons for this behavior.

EDIT: ...but given that this isn't the first time this has happened it seems like there is not much that can be done about it. I suppose you just can't have the same library loaded twice.

[stevengj]

Welcome to dependency hell: the operating system does not let you load two versions of the libstdc++ library simultaneously.

As far as I know, the the only solutions here are (1) recompile Julia to use the same C++ compiler as the Python library, or (2) recompile Scipy to use the same C++ compiler as Julia, or (3) use the LD_PRELOAD workaround.

[bhawkins]

I think I'm having a similar issue with other libraries. I have some python extension compiled against a conda stack that fails to find a symbol in libcurl when I try to pyimport it. To be more specific, I get

ImportError('/home/jovyan/.local/envs/isce3-dev/lib/libgdal.so.36: undefined symbol: curl_multi_poll')

I can use the following trick to see the list of libraries loaded by julia:

$ awk '/\.so/ {print $6}' /proc/$(pgrep julia)/maps | sort | uniq

For me this list includes libc and libstdc++, which are the headaches reported by others. However, after I run using PyCall the list gets longer by 40 libraries, including libcurl:

$ grep -i libcurl libs_julia_pycall.txt 
/home/jovyan/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/compiled/v1.11/LibCURL/ht49g_PBQaY.so
/home/jovyan/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/compiled/v1.11/LibCURL_jll/9JWaY_PBQaY.so
/usr/lib64/libcurl.so.4.5.0

It looks like some stuff has used name mangling, perhaps to avoid this very issue. But something is still pulling in the system libcurl which is older than the conda one and is missing necessary symbols. My GDAL library only specifies "libcurl.so.4", which I guess matches 4.5.0. The conda lib is libcurl.so.4.8.0, and GDAL has RPATH=$ORIGIN/., so it finds the correct one when you ldd it or import my package in Python.

I'm able to squelch the libcurl error by specifying the conda one with LD_PRELOAD, but then there's a similar issue with its own dependency libnghttp2 which is also loaded following using PyCall. If I specify both with LD_PRELOAD then I'm able to pyimport my package. I could also add a 4th option to @stevengj's list, which is that you can use patchelf to write the absolute paths to the library dependencies to the libraries themselves (like nix and spack do).

I'm not really surprised that you have to use a consistent libc and libstdc++, but I wasn't expecting a similar requirement for generic stuff like curl, zlib, and ssl. Maybe there's something particularly odd about my setup?

[bhawkins]

🦆 Answering my own question a day later, I see my system was configured with "/usr/lib64" in LD_LIBRARY_PATH, which makes "using PyCall" load it from there rather than its usual place ($HOME/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/lib/julia/libcurl.so.4.8.0). That happens to be the same soname version as my Conda stack, so removing "/usr/lib64" from my LD_LIBRARY_PATH solves my issue.

Edit: I'm still confused, though, because if I put my conda env lib directory ahead of /usr/lib64 then PyCall still loads libcurl from the latter. I must be missing something.