Remove libjulia-internal.so dependency on libstdc++ and libgcc #60248
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
This is great!
Can you show the stripped / non-stripped sizes? |
Member
Author
|
topolarity
reviewed
Nov 26, 2025
| # NB: CG needs uv_mutex_* symbols, but we expect to export them from libjulia-internal | ||
| CG_LIBS := $(LIBUNWIND) $(CG_LLVMLINK) $(OSLIBS) $(LIBTRACYCLIENT) $(LIBITTAPI) | ||
|
|
||
| ifeq ($(USEGCC),1) |
Member
There was a problem hiding this comment.
Is the USEGCC guard necessary?
I thought these flags were semi-standard at this point
Member
Author
There was a problem hiding this comment.
clang doesn't seem to mind if you pass -static-libstdc++ when linking to libc++ (seems to just ignore it), but it does error on -static-libgcc if we wouldn't normally link it.
topolarity
approved these changes
Nov 26, 2025
Member
|
Apparently this broke compilation here on Fedora 41 (gcc version 14.3.1): /usr/bin/ld: cannot find -lstdc++: No such file or directory
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:467: /home/milan/Dev/julia/usr/lib/libjulia-internal.so.1.14.0] Error 1
make: *** [Makefile:118: julia-src-release] Error 2It works with |
Member
Author
|
You are probably missing the |
Member
|
Ah of course. gcc could print a more explicit error... But yeah it would be worth adding to the docs. |
vtjnash
pushed a commit
that referenced
this pull request
Dec 2, 2025
Red Hat-based distros don't include the static version of libstdc++ in
their g++ packages, so document this dependency, as well as the
`USE_RT_STATIC_LIBSTDCXX=0` workaround if it isn't available. I also
looked for missing dependencies by compiling Julia in the base
`fedora:42` and `debian:12` images:
```dockerfile
FROM debian:12 AS build
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential libatomic1 python3 gfortran perl wget m4 cmake pkg-config \
curl git ca-certificates
WORKDIR /build
RUN git clone --depth=1 https://github.com/JuliaLang/julia.git
RUN cd julia && make -j binary-dist VERBOSE=1
```
```dockerfile
FROM fedora:42 AS build
RUN dnf install -y --nodocs --setopt=install_weak_deps=False \
gcc gcc-c++ gcc-gfortran python3 perl wget m4 cmake pkgconfig curl git \
which diffutils libatomic libstdc++-static
WORKDIR /build
RUN git clone --depth=1 https://github.com/JuliaLang/julia.git
RUN cd julia && make -j binary-dist VERBOSE=1
```
Reported by
#60248 (comment).
topolarity
added a commit
that referenced
this pull request
Dec 3, 2025
Somewhat of a companion to #60248. For a small application that has just started up `fork()` is not a huge concern, but it's quite heavy-handed for Julia- as-a-library scenarios where resident memory may already be large. Many soft-embedded targets also do not support fork() well, so it is good for our compatibility to adjust this. Rather than relying on the linker to do all of the heavy lifting, this changes our `libstdcxx` probe sequence to directly parse the `ld.so.cache` and `libstdc++.so.6` files. As long as we can expect `/etc/ld.so.cache` to be the same path on all Linux systems, this seems to be a reliable way to locate system libraries.
KristofferC
pushed a commit
that referenced
this pull request
Dec 4, 2025
Some testing by @BenChung has revealed how hard it is to load a JuliaC-built library into a program that has already loaded a very old version of libstdc++. Even with probing disabled, `libjulia-internal.so` fails because of missing GLIBCXX symbols. We use so little of the C++ standard library in `libjulia-internal.so` that it's worth the tradeoff to link it statically: it barely changes the size of the resulting library, removes a medium-size library we have to ship in trimmed bundles, and solves some of our hermeticity issues when being loaded by other software. `libjulia-codegen.so` uses it more extensively, and we expect to be able to load it as a plugin for `opt`, meaning it may have to remain dynamically linked. This PR contains a series of changes to enable statically linked libstdc++ by default and mitigate the size impact: - We enable `--gc-sections` when building with gcc/ld.bfd. This saves us some code already, since we can trim out a lot of libLLVMSupport/libLLVMTargetParser. On macOS, we can use `-dead_strip` to save some space even though the rest of these changes are not applicable. - Two flags for `-static-libstdc++` and `-static-libgcc`, called `USE_RT_STATIC_LIBSTDCXX` and `USE_RT_STATIC_LIBGCC`, are added and enabled by default. - Most of the additional code that gets linked into `libjulia-internal.so` is related to locales for iostreams. Replace these with standard C IO and LLVM helpers that don't have weird locale-related behaviour. - (NOT IMPLEMENTED) `--gc-sections` removes unused executable code, but leaves us with a lot of irrelevant debug info. I tested the effect of `llvm-dwarfutil --garbage-collection` and saw pretty good savings, but that is not included in this PR because BinaryBuilder doesn't have a new enough version of the LLVM tools. | Change | Size of `libjulia-internal.so` (KiB) | |--------------------------------------|--------------------------------------| | No change | 14524 | | Linker GC enabled | 13220 | | -static-libstdc++ and -static-libgcc | 22136 | | Excise iostreams | 15036 | | DWARF GC (not in this PR) | 11488 | This is a comparison of symbols that were added and removed. Even though 15 times more code is removed than added, the resulting `libjulia-internal.so` has a similar size to the original because of the additional debug info. (cherry picked from commit f36882f)
KristofferC
pushed a commit
that referenced
this pull request
Dec 4, 2025
Somewhat of a companion to #60248. For a small application that has just started up `fork()` is not a huge concern, but it's quite heavy-handed for Julia- as-a-library scenarios where resident memory may already be large. Many soft-embedded targets also do not support fork() well, so it is good for our compatibility to adjust this. Rather than relying on the linker to do all of the heavy lifting, this changes our `libstdcxx` probe sequence to directly parse the `ld.so.cache` and `libstdc++.so.6` files. As long as we can expect `/etc/ld.so.cache` to be the same path on all Linux systems, this seems to be a reliable way to locate system libraries. (cherry picked from commit ac4ee59)
KristofferC
pushed a commit
that referenced
this pull request
Dec 4, 2025
Some testing by @BenChung has revealed how hard it is to load a JuliaC-built library into a program that has already loaded a very old version of libstdc++. Even with probing disabled, `libjulia-internal.so` fails because of missing GLIBCXX symbols. We use so little of the C++ standard library in `libjulia-internal.so` that it's worth the tradeoff to link it statically: it barely changes the size of the resulting library, removes a medium-size library we have to ship in trimmed bundles, and solves some of our hermeticity issues when being loaded by other software. `libjulia-codegen.so` uses it more extensively, and we expect to be able to load it as a plugin for `opt`, meaning it may have to remain dynamically linked. This PR contains a series of changes to enable statically linked libstdc++ by default and mitigate the size impact: - We enable `--gc-sections` when building with gcc/ld.bfd. This saves us some code already, since we can trim out a lot of libLLVMSupport/libLLVMTargetParser. On macOS, we can use `-dead_strip` to save some space even though the rest of these changes are not applicable. - Two flags for `-static-libstdc++` and `-static-libgcc`, called `USE_RT_STATIC_LIBSTDCXX` and `USE_RT_STATIC_LIBGCC`, are added and enabled by default. - Most of the additional code that gets linked into `libjulia-internal.so` is related to locales for iostreams. Replace these with standard C IO and LLVM helpers that don't have weird locale-related behaviour. - (NOT IMPLEMENTED) `--gc-sections` removes unused executable code, but leaves us with a lot of irrelevant debug info. I tested the effect of `llvm-dwarfutil --garbage-collection` and saw pretty good savings, but that is not included in this PR because BinaryBuilder doesn't have a new enough version of the LLVM tools. | Change | Size of `libjulia-internal.so` (KiB) | |--------------------------------------|--------------------------------------| | No change | 14524 | | Linker GC enabled | 13220 | | -static-libstdc++ and -static-libgcc | 22136 | | Excise iostreams | 15036 | | DWARF GC (not in this PR) | 11488 | This is a comparison of symbols that were added and removed. Even though 15 times more code is removed than added, the resulting `libjulia-internal.so` has a similar size to the original because of the additional debug info. (cherry picked from commit f36882f)
KristofferC
pushed a commit
that referenced
this pull request
Dec 4, 2025
Somewhat of a companion to #60248. For a small application that has just started up `fork()` is not a huge concern, but it's quite heavy-handed for Julia- as-a-library scenarios where resident memory may already be large. Many soft-embedded targets also do not support fork() well, so it is good for our compatibility to adjust this. Rather than relying on the linker to do all of the heavy lifting, this changes our `libstdcxx` probe sequence to directly parse the `ld.so.cache` and `libstdc++.so.6` files. As long as we can expect `/etc/ld.so.cache` to be the same path on all Linux systems, this seems to be a reliable way to locate system libraries. (cherry picked from commit ac4ee59)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some testing by @BenChung has revealed how hard it is to load a JuliaC-built library into a program that has already loaded a very old version of libstdc++. Even with probing disabled,
libjulia-internal.sofails because of missing GLIBCXX symbols.We use so little of the C++ standard library in
libjulia-internal.sothat it's worth the tradeoff to link it statically: it barely changes the size of the resulting library, removes a medium-size library we have to ship in trimmed bundles, and solves some of our hermeticity issues when being loaded by other software.libjulia-codegen.souses it more extensively, and we expect to be able to load it as a plugin foropt, meaning it may have to remain dynamically linked.This PR contains a series of changes to enable statically linked libstdc++ by default and mitigate the size impact:
--gc-sectionswhen building with gcc/ld.bfd. This saves us some code already, since we can trim out a lot of libLLVMSupport/libLLVMTargetParser. On macOS, we can use-dead_stripto save some space even though the rest of these changes are not applicable.-static-libstdc++and-static-libgcc, calledUSE_RT_STATIC_LIBSTDCXXandUSE_RT_STATIC_LIBGCC, are added and enabled by default.libjulia-internal.sois related to locales for iostreams. Replace these with standard C IO and LLVM helpers that don't have weird locale-related behaviour.--gc-sectionsremoves unused executable code, but leaves us with a lot of irrelevant debug info. I tested the effect ofllvm-dwarfutil --garbage-collectionand saw pretty good savings, but that is not included in this PR because BinaryBuilder doesn't have a new enough version of the LLVM tools.libjulia-internal.so(KiB)This is a comparison of symbols that were added and removed. Even though 15 times more code is removed than added, the resulting
libjulia-internal.sohas a similar size to the original because of the additional debug info.