Partly revert "Reduce consequences of method invalidation for Dict and Set"

[JeffBezanson]

This small change had a very significant impact on build times and sysimg size. It must be because it forces us to specialize on all value types.

master:

Base  ─────────── 44.928756 seconds
Total ─────── 110.037750 seconds 
Generating precompile statements... 1977 generated in 156.256105 seconds (overhead 106.552134 seconds)
-rwxr-xr-x 1 jeff jeff 157101656 May  5 19:32 sys.so

with the commit reverted:

Base  ─────────── 39.333038 seconds
Total ───────  95.018324 seconds 
Generating precompile statements... 1754 generated in 147.468238 seconds (overhead 101.685924 seconds)
-rwxr-xr-x 1 jeff jeff 149454696 May  5 19:40 sys.so

The Dict setindex! method seems to be the vast majority of it; the new copymutable method is probably ok.

@timholy Could you check how much impact this has on the latencies you were testing?

[timholy]

Ugh, I'm sure that was fun to track down...

Here's current master (097f0b0):

julia> using Pkg

julia> @time try Pkg.add("NoPkg") catch err end
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `git@github.com:JuliaRegistries/General.git`
   Updating registry at `~/.julia/registries/HolyLabRegistry`
   Updating git-repo `git@github.com:HolyLab/HolyLabRegistry.git`
  2.675233 seconds (897.72 k allocations: 53.029 MiB, 0.38% gc time)

julia> @time try Pkg.add("NoPkg") catch err end
  0.019300 seconds (103.17 k allocations: 7.447 MiB)

julia> @time try Pkg.add("NoPkg") catch err end
  0.020904 seconds (103.17 k allocations: 7.447 MiB)

julia> using Revise

julia> @time try Pkg.add("NoPkg") catch err end
  0.014487 seconds (104.50 k allocations: 7.503 MiB)

Here's this branch:

julia> using Pkg

julia> @time try Pkg.add("NoPkg") catch err end
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `git@github.com:JuliaRegistries/General.git`
   Updating registry at `~/.julia/registries/HolyLabRegistry`
   Updating git-repo `git@github.com:HolyLab/HolyLabRegistry.git`
  3.156728 seconds (1.00 M allocations: 58.134 MiB, 0.31% gc time)

julia> @time try Pkg.add("NoPkg") catch err end
  0.013905 seconds (103.39 k allocations: 7.453 MiB)

julia> @time try Pkg.add("NoPkg") catch err end
  0.013872 seconds (103.39 k allocations: 7.453 MiB)

julia> using Revise

julia> @time try Pkg.add("NoPkg") catch err end
  1.292772 seconds (239.53 k allocations: 14.378 MiB)

julia> @time try Pkg.add("NoPkg") catch err end
  0.020901 seconds (103.39 k allocations: 7.453 MiB)

We're keeping the majority of the improvements from #35714, but it is a nontrivial regression.

BUT, I decided to poke at this a bit. I turns out that if I cherry-pick 5c5ab49 onto this branch, then I get

julia> using Pkg

julia> @time try Pkg.add("NoPkg") catch err end
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `git@github.com:JuliaRegistries/General.git`
   Updating registry at `~/.julia/registries/HolyLabRegistry`
   Updating git-repo `git@github.com:HolyLab/HolyLabRegistry.git`
  3.123242 seconds (997.75 k allocations: 58.035 MiB, 0.32% gc time)

julia> @time try Pkg.add("NoPkg") catch err end
  0.022758 seconds (103.38 k allocations: 7.453 MiB)

julia> @time try Pkg.add("NoPkg") catch err end
  0.020012 seconds (103.38 k allocations: 7.453 MiB)

julia> using Revise

julia> @time try Pkg.add("NoPkg") catch err end
  0.014153 seconds (104.72 k allocations: 7.510 MiB)

So it's just a question of this branch not having all the other good things that master has. I think I made the Dict change before implementing JuliaLang/Pkg.jl#1806, and with the latter it seems the former isn't crucial.

I think we can merge this. Sorry that change ended up being costly while also not actually improving things.

[JeffBezanson]

Great, that's a relief!