Generate precompile directives via PrecompileSignatures.jl

[rikhuijzer]

After manually adding a lot of precompile directives in earlier PRs such as the following:

• Use precompile for SessionActions.open #1934
• 🧂 more precompile for dynamic functions! #1978

it seemed that there should be a better way. What if we could automatically generate precompile directives? Of course, guessing what types are going to be put into functions is impossible to predict without actually running code and keeping track of calls via SnoopCompile.jl, but what we can do is read the method signatures from the package and generate precompile directives for any concrete signatures. Yes, it's a very brute force way, but if it works it works.

This PR demonstrates the new and yet unregistered PrecompileSignatures.jl package. This package reads the method signatures inside Pluto.jl and generates about 415 precompile directives from that.

Benchmarks

With Julia 1.8-beta3:

main

Precompilation time: 8 seconds

 Section                        ncalls     time    %tot     alloc    %tot
 ────────────────────────────────────────────────────────────────────────
 import Pluto                        1    562ms    1.9%   74.6MiB    3.0%
 compiletimes.jl                     1    29.3s   98.1%   2.38GiB   97.0%
   PlutoRunner.run_expression        1    1.41s    4.7%   79.3MiB    3.2%
   SessionActions.open               1    17.9s   59.9%   1.31GiB   53.4%
   Pluto.run                         1    4.80s   16.1%    563MiB   22.4%
 ──────────────────────────────────────────────────────────────────────── 

This PR

Precompilation time: 12 seconds

 Section                        ncalls     time    %tot     alloc    %tot
 ────────────────────────────────────────────────────────────────────────
 import Pluto                        1    1.24s    4.4%    151MiB    7.5%
 compiletimes.jl                     1    27.1s   95.6%   1.81GiB   92.5%
   PlutoRunner.run_expression        1    1.52s    5.4%   78.0MiB    3.9%
   SessionActions.open               1    16.2s   57.1%    889MiB   44.3%
   Pluto.run                         1    4.31s   15.2%    441MiB   22.0%
 ────────────────────────────────────────────────────────────────────────

EDIT: Earlier versions of this comment showed less promising results. This was due to me enthusiastically removing all manually added precompile directives; even the ones which couldn't be automatically added by PrecompileSignatures.jl.

[github-actions]

Try this Pull Request!

Open Julia and type:

julia> import Pkg
julia> Pkg.activate(temp=true)
julia> Pkg.add(url="https://github.com/fonsp/Pluto.jl", rev="rh/precompilesignatures")
julia> using Pluto

[rikhuijzer]

@fonsp @Pangoraw I think this is good to go. Can you review? I've added 32-bit tests at rikhuijzer/PrecompileSignatures.jl#3 and those pass too.

I haven't tested package relocatability because I don't know how to test that. Help is welcome. I don't expect problems with that because the precompile directives are generated automatically, but better safe than sorry I guess.

I have also tested the package now in PlutoStaticHTML.jl and triggered a CI run on JuliaTutorialsTemplate. Also there everything works as it should.

[fonsp]

Does https://github.com/rikhuijzer/PrecompileSignatures.jl use internal Julia API? Like ccall(:jl_generating_output, Cint, ())?

If so, can you wrap this in a try-catch, or something similar, to avoid issues with running Pluto on future Julia versions?

[rikhuijzer]

Does https://github.com/rikhuijzer/PrecompileSignatures.jl use internal Julia API? Like ccall(:jl_generating_output, Cint, ())?

If so, can you wrap this in a try-catch, or something similar, to avoid issues with running Pluto on future Julia versions?

The ccall is used by packages like DataFrames.jl and Revise.jl too.

Still, you made a good point. So, in version two of PrecompileSignatures.jl I simplified the API to one single macro @precompile_signatures(Pluto). Also CI runs on Julia 1.6 and nightly. This macro contains a try-catch to avoid breaking stuff:

https://github.com/rikhuijzer/PrecompileSignatures.jl/blob/656a6c13835f5fdb4718b0ece476644ce66bca7d/src/PrecompileSignatures.jl#L253-L269

[rikhuijzer]

And another benchmark to be surer. This time on my laptop with Julia 1.7.3

main

 Section                        ncalls     time    %tot     alloc    %tot
 ────────────────────────────────────────────────────────────────────────
 import Pluto                        1    690ms    1.9%   76.1MiB    2.2%
 compiletimes.jl                     1    35.3s   98.1%   3.23GiB   97.8%
   PlutoRunner.run_expression        1    1.70s    4.7%    128MiB    3.8%
   SessionActions.open               1    23.6s   65.5%   2.08GiB   63.0%
   Pluto.run                         1    4.86s   13.5%    581MiB   17.2%
 ──────────────────────────────────────────────────────────────────────── 

This PR

 Section                        ncalls     time    %tot     alloc    %tot
 ────────────────────────────────────────────────────────────────────────
 import Pluto                        1    1.26s    3.7%    156MiB    5.2%
 compiletimes.jl                     1    32.4s   96.3%   2.79GiB   94.8%
   PlutoRunner.run_expression        1    1.55s    4.6%    127MiB    4.2%
   SessionActions.open               1    21.2s   63.0%   1.74GiB   59.1%
   Pluto.run                         1    4.44s   13.2%    481MiB   16.0%
 ────────────────────────────────────────────────────────────────────────

Really promising speed gains. NB. It is also promising that this setup extends the toolbox when working on TTFX.

[fonsp]

Awesome, thanks for the changes! Also see rikhuijzer/PrecompileSignatures.jl#8