Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lazy JLLs #50687

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Lazy JLLs #50687

wants to merge 4 commits into from

Conversation

staticfloat
Copy link
Member

@staticfloat staticfloat commented Jul 27, 2023 •
edited
Loading

This PR converts all of our stdlib JLLs to be "lazy" JLLs, which provides a nice improvement to startup latency, and reduces overall memory footprint by not loading libraries that are never called.

Example timings:

# current master
$ hyperfine -w 1 -r 20 ./julia
Benchmark 1: ./julia
  Time (mean ± σ):     152.2 ms ±   1.8 ms    [User: 272.5 ms, System: 86.7 ms]
  Range (min … max):   149.3 ms … 155.3 ms    20 runs

# this PR
$ hyperfine -w 1 -r 20 ./julia
Benchmark 1: ./julia
  Time (mean ± σ):     122.4 ms ±   2.3 ms    [User: 86.7 ms, System: 33.2 ms]
  Range (min … max):   119.6 ms … 130.4 ms    20 runs

Depends on:

@staticfloat staticfloat mentioned this pull request Jul 27, 2023
Open
@oscardssmith
Copy link
Member

what's the rss difference?

@staticfloat
Showcase example of using LazyLibrary in shipped JLLs
7df0d9c 
This converts the following JLLs to LazyLibrary APIs:

* LibCURL_jll
* LibSSH2_jll
* MbedTLS_jll
* OpenBLAS_jll
* libblastrampoline_jll
* nghttp2_jll

It also introduces a new precompilation statement for a basic matrix
multiplciation to ensure that TTFMM (time to first matmul) remains
excellent.
@staticfloat
More lazy JLL stdlibs
9ebc52e 
* `OpenBLAS_jll`
* `CompilerSupportLibraries_jll`
* `dSFMT_jll`
@staticfloat
Change Random to use dSFMT_jll
30dfa21
@staticfloat
Copy link
Member Author

what's the rss difference?

Not much, 2 MB out of 190 MB. I'm a little surprised by that, but I think it's likely because most of the pages we're eliminating here are shared.

@staticfloat staticfloat changed the title [WIP] Lazy JLLs Lazy JLLs Jul 27, 2023
@staticfloat staticfloat marked this pull request as ready for review July 27, 2023 15:32
@staticfloat staticfloat requested a review from vtjnash July 27, 2023 15:36
fingolfin
fingolfin reviewed Jul 27, 2023
View reviewed changes
Copy link
Contributor

@fingolfin fingolfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice :-)

base/Base.jl Show resolved Hide resolved
base/base_jll_adapters.jl Show resolved Hide resolved
base/base_jll_adapters.jl Show resolved Hide resolved
base/base_jll_insertion.jl Show resolved Hide resolved
base/gmp.jl Show resolved Hide resolved
base/gmp.jl Outdated Show resolved Hide resolved
base/pcre.jl Show resolved Hide resolved
stdlib/PCRE2_jll/src/PCRE2_jll.jl Show resolved Hide resolved
@staticfloat
Lazy base JLLs
12269e0 
This implements lazy libraries for the three potentially-lazy libraries
needed by Base, `PCRE2_jll`, `GMP_jll` and `MPFR_jll`.  Because these
libraries are needed by bootstrap (and in the case of `PCRE2`, needed by
Julia's own initialization) we have to go through some extra work to get
them to properly load lazily.

We move the `LazyLibraryPath` definition to be much earlier in the
bootstrap process and use this path to load e.g. `GMP` and `MPFR` during
bootstrap, but then replace that path with a `LazyLibrary` after
bootstrap has finished.  This provides a mechanism for loading things
before e.g. `dlopen()` has been defined.
@PallHaraldsson
Copy link
Contributor

PallHaraldsson commented Jul 28, 2023 •
edited
Loading

converts all of our stdlib JLLs to be "lazy" JLLs, which provides a nice improvement to startup latency

This is for stdlib JLLs only, right? Still great, but the idea would translate to all JLLs? Easy to extend that way, and might also make for nice speedup? Each one is seemingly usually very cheap, but sometimes many used, so add up. I was also (alternatively) thinking parallel loading of JLLs, seems doable, probably easier than for non-JLL.

Is this too large to backport? Not right away, but ion time for 1.10?

@staticfloat
Copy link
Member Author

This is for stdlib JLLs only, right? Still great, but the idea would translate to all JLLs?

Yes, that's the idea, but we need more information than exists in JLLs right now; we need to encode the dependency graph of each library on every other library.

Is this too large to backport? Not right away, but ion time for 1.10?

I don't think this is a huge feature, and my time to work on it post-JuliaCon will be limited, so I would expect it in 1.11.

@vtjnash vtjnash mentioned this pull request Aug 1, 2023
Merged
@staticfloat staticfloat mentioned this pull request Sep 13, 2023
Merged
@topolarity topolarity mentioned this pull request Aug 12, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fingolfin fingolfin fingolfin left review comments

@vtjnash vtjnash Awaiting requested review from vtjnash

Assignees
No one assigned
Labels
JLLs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@staticfloat @oscardssmith @PallHaraldsson @fingolfin @brenhinkeller