Loading unrelated packages makes functions take >30x as long on first run

[oxinabox]

See invenia/JLSO.jl#42 by @colinxs

With JLSO@v1.3.0 and StaticArrays@v0.12.1 I'm seeing the following:

julia> using JLSO
julia> open("/tmp/foo.jlso", "w") do io
            write(io, JLSOFile(:data => rand(1000)))
          end
julia> @time JLSO.load("/tmp/foo.jlso")
  5.034648 seconds (16.27 M allocations: 781.737 MiB, 7.60% gc time)

vs

julia> using StaticArrays, JLSO
julia> open("/tmp/foo.jlso", "w") do io
            write(io, JLSOFile(:data => rand(1000)))
          end
julia> @time JLSO.load("/tmp/foo.jlso")
  172.650491 seconds (259.65 M allocations: 16.296 GiB, 4.05% gc time)

Second call to JLSO.load is still fast, however. Here's my versioninfo() but I'm seeing this across multiple machines.

Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, ivybridge)

Let me know if this issue should be moved to StaticArrays and I'll move it over.

I have verified this and see the same

[JeffBezanson]

Seems like the exact same sort of issue as #34098.

[KristofferC]

It should be noted that invenia/JLSO.jl#44 completely fixes this.

Anyway, using the script posted earlier:

julia> badpath(stdout, G, Tuple{typeof(JLSO.load),String})
Tuple{typeof(JLSO.load),String} 13989
Tuple{typeof(open),JLSO.var"#37#38"{Tuple{}},String} 13988
Tuple{Base.var"##open#275",Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},typeof(open),JLSO.var"#37#38"{Tuple{}},String} 13985
Tuple{JLSO.var"#37#38"{Tuple{}},IOStream} 13978
Tuple{typeof(JLSO.load),IOStream} 13977
Tuple{typeof(read),IOStream,Type{JLSOFile}} 13499
Tuple{typeof(JLSO.upgrade_jlso),AbstractDict} 11423
Tuple{typeof(JLSO.upgrade_jlso),AbstractDict,Val{2}} 11088
Tuple{typeof(JLSO._upgrade_env),Dict} 10585
Tuple{typeof(mktempdir),JLSO.var"#20#24"{_A} where _A} 10581
Tuple{typeof(mktempdir),JLSO.var"#20#24"{_A} where _A,String} 10579
Tuple{Base.Filesystem.var"##mktempdir#20",String,typeof(mktempdir),JLSO.var"#20#24"{_A} where _A,String} 10578
Tuple{JLSO.var"#20#24"{_A} where _A,String} 10561
Tuple{typeof(Pkg.API.add),Union{AbstractString, Pkg.Types.PackageSpec}} 10206
Tuple{Pkg.API.var"##add#23",Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},typeof(Pkg.API.add),AbstractString} 10206
Tuple{typeof(Pkg.API.add),Array{#s662,1} where #s662<:AbstractString} 10206
Tuple{Pkg.API.var"##add#24",Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},typeof(Pkg.API.add),Array{#s662,1} where #s662<:AbstractString} 10206
Tuple{typeof(Pkg.API.add),Array{Pkg.Types.PackageSpec,1}} 9744
Tuple{Pkg.API.var"##add#27",Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},typeof(Pkg.API.add),Array{Pkg.Types.PackageSpec,1}} 9741
Tuple{typeof(Pkg.API.add),Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1}} 9736
Tuple{Pkg.API.var"##add#28",Pkg.Types.PreserveLevel,Pkg.BinaryPlatforms.Platform,Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},typeof(Pkg.API.add),Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1}} 9732
Tuple{Pkg.Operations.var"#add##kw",NamedTuple{(:preserve, :platform),_A} where _A<:Tuple,typeof(Pkg.Operations.add),Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1},Array{Base.UUID,1}} 9664
Tuple{Pkg.Operations.var"##add#71",Pkg.Types.PreserveLevel,Pkg.BinaryPlatforms.Platform,typeof(Pkg.Operations.add),Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1},Array{Base.UUID,1}} 9649
Tuple{typeof(Pkg.Operations.build_versions),Pkg.Types.Context,Array{Base.UUID,1}} 9506
Tuple{Pkg.Operations.var"##build_versions#56",Bool,Bool,typeof(Pkg.Operations.build_versions),Pkg.Types.Context,Array{Base.UUID,1}} 9506
Tuple{typeof(Pkg.Operations.sandbox),Pkg.Operations.var"#59#63"{Bool,Pkg.Types.Context,String,Pkg.Types.PackageSpec,String},Pkg.Types.Context,Pkg.Types.PackageSpec,String,String,Nothing} 9506
...

So it seems it goes into Pkg and recompiles everything there?

[oxinabox]

It should be noted that invenia/JLSO.jl#44 completely fixes this.

It should also be notes that invenia/JLSO.jl#44 is very unidiomatic.
Directly goes against general guideline of "putting type constraints on functions doesn't change anything"

[KristofferC]

Don't really understand what you mean, it just removes code?

[JeffBezanson]

Unfortunately compiler performance can't be included in any guarantees about which programs are equivalent.

[oxinabox]

Don't really understand what you mean, it just removes code?

Ah, sorry I ment invenia/JLSO.jl#43
which is an unidomatic change, and was added to fix it