Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long import times for Chains during testing #171

Closed
itsdfish opened this issue Jan 31, 2020 · 33 comments
Closed

Long import times for Chains during testing #171

itsdfish opened this issue Jan 31, 2020 · 33 comments

Comments

@itsdfish
Copy link
Contributor

I am opening this issue to document long import times for Chains. In some cases, it can take upwards of an hour locally, causing times outs on travis.

In the following MWE with fewer dependencies, the tests complete faster, but are nonetheless very long.

On my local machine, I have found that calling import MCMCChains: Chains slows down testing significantly, and the slow down is particularly pronounced with Requires 1.0 on the first run. This finding is particularly important for testing on travis because it uses a fresh installation and will timeout in larger packages. Here are my results:

# First Run 
#                No Chains  Chains
#Requires 0.5:   1.82        14.92
#Requires 1.0:   1.55       129.78

# Second Run
#                No Chains  Chains
#Requires 0.5:   0.72       07.06
#Requires 1.0:   0.71       15.41

The general pattern has been reproduced on travis:

With the import statement: about 1 minute.
Without the import statement: about 7 minutes.

@devmotion
Copy link
Member

devmotion commented Jan 31, 2020

Have to tried to downgrade MCMCChains? Would be interesting to see if one can figure out if it is caused by a specific combination of packages and if a specific commit caused a performance regression.

@itsdfish
Copy link
Contributor Author

itsdfish commented Jan 31, 2020

Good point. I was able to reproduce that pattern of results with MCMCChains 0.4.1. MCMCChains .3.4 had the same pattern of results but ran at 51 seconds with Requires 1.0, instead of 130ish. I could not run earlier versions without errors.

@devmotion
Copy link
Member

Could you post the full list of packages (output of ] st --manifest) that were installed and used in these runs?

@itsdfish
Copy link
Contributor Author

itsdfish commented Jan 31, 2020

For MCMCChains 3.4:

(v1.3) pkg> st --manifest
    Status `~/.julia/environments/v1.3/Manifest.toml`
  [7d9fca2a] Arpack v0.4.0
  [68821587] Arpack_jll v3.5.0+2
  [39de3d68] AxisArrays v0.4.0
  [b99e7846] BinaryProvider v0.5.8
  [9a962f9c] DataAPI v1.1.0
  [864edb3b] DataStructures v0.17.9
  [31c24e10] Distributions v0.22.4
  [da5c29d0] EllipsisNotation v0.4.0
  [1a297f60] FillArrays v0.8.4
  [bae3c9db] HangTest v0.1.0 [`~/.julia/dev/HangTest`]
  [8197267c] IntervalSets v0.4.0
  [c8e1da08] IterTools v1.3.0
  [c7f686f2] MCMCChains v0.3.4
  [e1d29d7a] Missings v0.4.3
  [4536629a] OpenBLAS_jll v0.3.7+5
  [efe28fd5] OpenSpecFun_jll v0.5.3+1
  [bac558e1] OrderedCollections v1.1.0
  [90014a1f] PDMats v0.9.11
  [1fd47b50] QuadGK v2.3.1
  [b3c3ace0] RangeArrays v0.3.2
  [3cdcf5f2] RecipesBase v0.7.0
  [ae029012] Requires v1.0.0
  [79098fc4] Rmath v0.6.0
  [992d4aef] Showoff v0.3.1
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.9.0
  [2913bbd2] StatsBase v0.32.0
  [4c63d2b9] StatsFuns v0.9.3
  [2a0f44e3] Base64 
  [ade2ca70] Dates 
  [8ba89e20] Distributed 
  [b77e0a4c] InteractiveUtils 
  [76f85450] LibGit2 
  [8f399da3] Libdl 
  [37e2e46d] LinearAlgebra 
  [56ddb016] Logging 
  [d6f4376e] Markdown 
  [44cfe95a] Pkg 
  [de0858da] Printf 
  [3fa0cd96] REPL 
  [9a3f8284] Random 
  [ea8e919c] SHA 
  [9e88b42a] Serialization 
  [6462fe0b] Sockets 
  [2f01184e] SparseArrays 
  [10745b16] Statistics 
  [4607b0f0] SuiteSparse 
  [8dfed614] Test 
  [cf7118a7] UUIDs 
  [4ec0a83e] Unicode 

For MCMCChains .4.1

    Status `~/.julia/environments/v1.3/Manifest.toml`
  [621f4979] AbstractFFTs v0.5.0
  [80f14c24] AbstractMCMC v0.1.0
  [7d9fca2a] Arpack v0.4.0
  [68821587] Arpack_jll v3.5.0+2
  [4fba245c] ArrayInterface v2.3.1
  [13072b0f] AxisAlgorithms v1.0.0
  [39de3d68] AxisArrays v0.3.3
  [b99e7846] BinaryProvider v0.5.8
  [324d7699] CategoricalArrays v0.7.7
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v2.2.0
  [9a962f9c] DataAPI v1.1.0
  [a93c6f00] DataFrames v0.20.0
  [864edb3b] DataStructures v0.17.9
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.0.2
  [b552c78f] DiffRules v1.0.0
  [31c24e10] Distributions v0.21.12
  [da5c29d0] EllipsisNotation v0.4.0
  [7a1cc6ca] FFTW v1.2.0
  [f5851436] FFTW_jll v3.3.9+3
  [1a297f60] FillArrays v0.8.4
  [6a86dc24] FiniteDiff v2.1.0
  [f6369f11] ForwardDiff v0.10.9
  [bae3c9db] HangTest v0.1.0 [`~/.julia/dev/HangTest`]
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+0
  [a98d9a8b] Interpolations v0.12.8
  [8197267c] IntervalSets v0.4.0
  [41ab1584] InvertedIndices v1.0.0
  [c8e1da08] IterTools v1.3.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [682c06a0] JSON v0.21.0
  [5ab0869b] KernelDensity v0.5.1
  [d3d80556] LineSearches v7.0.1
  [c7f686f2] MCMCChains v0.4.1
  [856f044c] MKL_jll v2019.0.117+2
  [e1d29d7a] Missings v0.4.3
  [d41bc354] NLSolversBase v7.6.1
  [77ba4419] NaNMath v0.3.3
  [6fe1bfb0] OffsetArrays v1.0.2
  [4536629a] OpenBLAS_jll v0.3.7+5
  [efe28fd5] OpenSpecFun_jll v0.5.3+1
  [429524aa] Optim v0.20.1
  [bac558e1] OrderedCollections v1.1.0
  [90014a1f] PDMats v0.9.11
  [d96e819e] Parameters v0.12.0
  [69de0a69] Parsers v0.3.10
  [2dfb63ee] PooledArrays v0.5.3
  [85a6dd25] PositiveFactorizations v0.2.3
  [92933f4c] ProgressMeter v1.2.0
  [1fd47b50] QuadGK v2.3.1
  [b3c3ace0] RangeArrays v0.3.2
  [c84ed2f1] Ratios v0.3.1
  [3cdcf5f2] RecipesBase v0.7.0
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v1.0.0
  [79098fc4] Rmath v0.6.0
  [992d4aef] Showoff v0.3.1
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.9.0
  [90137ffa] StaticArrays v0.12.1
  [2913bbd2] StatsBase v0.32.0
  [4c63d2b9] StatsFuns v0.9.3
  [3783bdb8] TableTraits v1.0.0
  [bd369af6] Tables v0.2.11
  [efce3f68] WoodburyMatrices v0.5.0
  [2a0f44e3] Base64 
  [ade2ca70] Dates 
  [8bb1440f] DelimitedFiles 
  [8ba89e20] Distributed 
  [9fa8497b] Future 
  [b77e0a4c] InteractiveUtils 
  [76f85450] LibGit2 
  [8f399da3] Libdl 
  [37e2e46d] LinearAlgebra 
  [56ddb016] Logging 
  [d6f4376e] Markdown 
  [a63ad114] Mmap 
  [44cfe95a] Pkg 
  [de0858da] Printf 
  [3fa0cd96] REPL 
  [9a3f8284] Random 
  [ea8e919c] SHA 
  [9e88b42a] Serialization 
  [1a1011a3] SharedArrays 
  [6462fe0b] Sockets 
  [2f01184e] SparseArrays 
  [10745b16] Statistics 
  [4607b0f0] SuiteSparse 
  [8dfed614] Test 
  [cf7118a7] UUIDs 

@devmotion
Copy link
Member

Are you sure that's the packages for both runs? It says [c7f686f2] MCMCChains v0.3.4 in both cases.

@itsdfish
Copy link
Contributor Author

Sorry about the confusion. It looks like I repasted the first list because I did not copy correctly. I updated the list above.

@devmotion
Copy link
Member

Thanks! Just had a quick look at the lists, but I'm quite surprised that version 0.4.1 seems to use older versions of the dependencies such as AxisArrays and Distributions. That seems weird?!

@devmotion
Copy link
Member

And the list of dependencies seems to have grown quite significantly...

@devmotion
Copy link
Member

Maybe that's mainly since DataFrames (and KernelDensity) were added as dependencies between 0.3.4 and 0.4.1:
v0.3.4...v0.4.1
I guess it would be nice not having to depend on DataFrames. There is some discussion about separating the type definitions and constructors into a lightweight package (JuliaData/DataFrames.jl#1764), since apparently it can take quite some time to load and has quite some dependencies (https://discourse.julialang.org/t/sluggish-performance-when-loading-in-modules-dataframes-csv/17857). Maybe MCMCChains could get around this by only implementing the Tables interface? I guess then one might be able to get conversions to DataFrame for free.

@itsdfish
Copy link
Contributor Author

Sometimes the versions depend on the order in which they are installed and the requirements of the other dependencies. That is why I installed HangTest within a new environment. So it is not uncommon to see dependency downgrades.

@devmotion
Copy link
Member

I just noticed that Requires is not a direct requirement of MCMCChains - do you happen to know which of the dependencies pulls it in? I'm wondering if that indicates that it's not (only) an MCMCChains problem but (also) a problem of one of its dependencies, in particular when thinking about the different timings with Requires 0.5 and 1.0.

@goedman
Copy link
Collaborator

goedman commented Jan 31, 2020

This has been around since Julia 1.3. Over the last 2 months I've spend almost all my cycles on trying to figure out what is happening and came to the conclusion that it is a combination of figuring out package versions and many dependencies. Why it suddenly degraded in Julia 1.3 is unclear to me.

During the last 2 months I removed the fixed dependency on MCMCChains from CmdStan.jl and Stan.jl and replaced it by using Requires.jl. This solved the immediate problem (very long compile times, travis timeouts and docs on travis no longer building and even what I once erroneously called stray processes on JuliaLang). It has blown me away that I've seen so little about this issue mentioned.

@goedman
Copy link
Collaborator

goedman commented Jan 31, 2020

By the way, over the last week I tested quite a bit with DataFrames.jl and have never seen issues with that package.

@itsdfish
Copy link
Contributor Author

itsdfish commented Jan 31, 2020

@devmotion, I'm not sure if there are dependencies in MCMCChains that currently have restrictions on Requires. However, a more general problem is that other packages that people use will probably have constraints.

@goedman, I have been dealing with this issue for a while also. I just haven't had the time to pinpoint what was causing the issue. It took me a while to narrow down the problem to MCMCChains. I was forced to investigate the issue more thoroughly when it caused the MCMC sampler I coded for AdvancedPS to timeout on travis. I suppose some people might be ignoring the issue until it presents an impasse.

@goedman
Copy link
Collaborator

goedman commented Jan 31, 2020

To test MCMCChains, I typically activate MCMCChains as a project but that didn't give a lot of new insights:

Activating environment at `~/.julia/dev/MCMCChains/Project.toml`
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-DEV.212 (2020-01-31)
 _/ |\__'_|_|_|\__'_|  |  Commit 51f1710d7e (0 days old master)
|__/                   |

(MCMCChains) pkg> up
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Updating `~/.julia/dev/MCMCChains/Project.toml`
  [80f14c24] + AbstractMCMC v0.1.0
  [39de3d68] + AxisArrays v0.3.3
  [a93c6f00] + DataFrames v0.20.0
  [31c24e10] + Distributions v0.21.12
  [5ab0869b] + KernelDensity v0.5.1
  [3cdcf5f2] + RecipesBase v0.7.0
  [992d4aef] + Showoff v0.3.1
  [276daf66] + SpecialFunctions v0.9.0
  [2913bbd2] + StatsBase v0.32.0
  [37e2e46d] + LinearAlgebra 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [10745b16] + Statistics 
   Updating `~/.julia/dev/MCMCChains/Manifest.toml`
  [621f4979] + AbstractFFTs v0.5.0
  [80f14c24] + AbstractMCMC v0.1.0
  [7d9fca2a] + Arpack v0.4.0
  [68821587] + Arpack_jll v3.5.0+2
  [4fba245c] + ArrayInterface v2.3.1
  [13072b0f] + AxisAlgorithms v1.0.0
  [39de3d68] + AxisArrays v0.3.3
  [b99e7846] + BinaryProvider v0.5.8
  [324d7699] + CategoricalArrays v0.7.7
  [bbf7d656] + CommonSubexpressions v0.2.0
  [34da2185] + Compat v2.2.0
  [9a962f9c] + DataAPI v1.1.0
  [a93c6f00] + DataFrames v0.20.0
  [864edb3b] + DataStructures v0.17.9
  [e2d170a0] + DataValueInterfaces v1.0.0
  [163ba53b] + DiffResults v1.0.2
  [b552c78f] + DiffRules v1.0.0
  [31c24e10] + Distributions v0.21.12
  [da5c29d0] + EllipsisNotation v0.4.0
  [7a1cc6ca] + FFTW v1.2.0
  [f5851436] + FFTW_jll v3.3.9+3
  [1a297f60] + FillArrays v0.8.4
  [6a86dc24] + FiniteDiff v2.1.0
  [f6369f11] + ForwardDiff v0.10.9
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+0
  [a98d9a8b] + Interpolations v0.12.8
  [8197267c] + IntervalSets v0.4.0
  [41ab1584] + InvertedIndices v1.0.0
  [c8e1da08] + IterTools v1.3.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [682c06a0] + JSON v0.21.0
  [5ab0869b] + KernelDensity v0.5.1
  [d3d80556] + LineSearches v7.0.1
  [856f044c] + MKL_jll v2019.0.117+2
  [e1d29d7a] + Missings v0.4.3
  [d41bc354] + NLSolversBase v7.6.1
  [77ba4419] + NaNMath v0.3.3
  [6fe1bfb0] + OffsetArrays v1.0.2
  [4536629a] + OpenBLAS_jll v0.3.7+5
  [efe28fd5] + OpenSpecFun_jll v0.5.3+1
  [429524aa] + Optim v0.20.1
  [bac558e1] + OrderedCollections v1.1.0
  [90014a1f] + PDMats v0.9.11
  [d96e819e] + Parameters v0.12.0
  [69de0a69] + Parsers v0.3.10
  [2dfb63ee] + PooledArrays v0.5.3
  [85a6dd25] + PositiveFactorizations v0.2.3
  [92933f4c] + ProgressMeter v1.2.0
  [1fd47b50] + QuadGK v2.3.1
  [b3c3ace0] + RangeArrays v0.3.2
  [c84ed2f1] + Ratios v0.3.1
  [3cdcf5f2] + RecipesBase v0.7.0
  [189a3867] + Reexport v0.2.0
  [ae029012] + Requires v1.0.0
  [79098fc4] + Rmath v0.6.0
  [992d4aef] + Showoff v0.3.1
  [a2af1166] + SortingAlgorithms v0.3.1
  [276daf66] + SpecialFunctions v0.9.0
  [90137ffa] + StaticArrays v0.12.1
  [2913bbd2] + StatsBase v0.32.0
  [4c63d2b9] + StatsFuns v0.9.3
  [3783bdb8] + TableTraits v1.0.0
  [bd369af6] + Tables v0.2.11
  [efce3f68] + WoodburyMatrices v0.5.0
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [9fa8497b] + Future 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [1a1011a3] + SharedArrays 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [4607b0f0] + SuiteSparse 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 

(MCMCChains) pkg> precompile
Precompiling project...
[ Info: Precompiling Showoff [992d4aef-0814-514b-bc4d-f2e9a6c4116f]
[ Info: Precompiling Distributions [31c24e10-a181-5473-b8eb-7969acd0382f]
[ Info: Precompiling KernelDensity [5ab0869b-81aa-558d-bb23-cbf5423bbe9b]
[ Info: Precompiling AxisArrays [39de3d68-74b9-583c-8d2d-e117c070f3a9]
[ Info: Precompiling AbstractMCMC [80f14c24-f653-4e6a-9b94-39d6b0f70001]
[ Info: Precompiling RecipesBase [3cdcf5f2-1ef4-517c-9805-6587b60abb01]
[ Info: Precompiling DataFrames [a93c6f00-e57d-5684-b7b6-d8193f3e46c0]
[ Info: Precompiling MCMCChains [c7f686f2-ff18-58e9-bc7b-31028e88f75d]

Takes a couple of minutes on my machine.

Part of the issue is that MCMCChains depends on Plots.jl indirectly (via RecipeBase) which typically causes another long delay the first time a plot is needed (this is definitely being addressed I believe).

@cpfiffer
Copy link
Member

Someone on Slack recommend this post. MCMCChains shows up towards the end of the thread. I threw in some comments -- maybe Tim has some ideas.

The weirdest thing to me is that the issue only happens at module scope. using MCMCChains takes about as long as other big packages to load if you call it from the REPL, but HangTest takes 111 seconds on my machine and the only difference is scoping.

@cpfiffer
Copy link
Member

Possibly related, though more about REPL behavior:

JuliaLang/julia#34098

@goedman
Copy link
Collaborator

goedman commented Jan 31, 2020

Good links, Cameron. To me it looks like the root cause of the problem is still unclear and thus likely there are several factors in play. After activating HangTest,using HangTest doesn't take that long on my system but ] test hasn't returned for 9 minutes now. What I erroneously called stray processes.

I am very disappointed not to be able to always use MCMCChains any longer. It worked great in CmdStan 5.x and Julia < 1.3. I wonder if there would be interest, if possible, to refactor MCMCChains?

@cpfiffer
Copy link
Member

I am very disappointed not to be able to always use MCMCChains any longer. It worked great in CmdStan 5.x and Julia < 1.3. I wonder if there would be interest, if possible, to refactor MCMCChains?

Yes, there is tremendous interest, at least from me. I would basically tear the whole thing apart and start from ground zero, with some things kept in spirit or modified heavily. The diagnostic stuff should be generalized better and perhaps reduced to core functionality. The AxisArray stuff is clumsily done and would probably be better handled with a StructArray for the highly nonlinear data that Bayesian models often represent.

Unfortunately my initial refactoring many months ago was probably not destined to be the high-quality redesign I wanted. When I did the initial refactoring of MCMCChains, there were two factors that kept it from being as good as it could have been.

First, I am a much better developer now than I was at the time -- if I sat down with it again, there would probably be some significant improvements across the board. I would drastically reduce the dependency list, cut down on unnecessary functions, and generally slim everything down. MCMCChains is riddled with poor choices.

Second, there's a lot of legacy stuff in there that's left over from Klara (!) that really should have just been built from the ground up with a cohesive focus on performance. I was trying really hard to shoehorn a new thing into an old thing without thinking about how to adapt the old thing, and I think that has really started to show.

I should note that with the general methods in AbstractMCMC, there's no reason to rely on the monolithic repo that MCMCChains has become -- @devmotion and I are already sort of starting to build tools that allow users to define general chain types that can be specified. If you want plotting and stuff, use MCMCChains. If you want performance, maybe you want a light StructArray wrapper with a small amount of metadata. Maybe you don't want any of this chain garbage and you just want a vector of raw output from the sampler, in which case we might want to think about general diagnostic tools for all <: AbstractChain.

We've been kicking around the idea of a LightMCMCChains repo (name may change) that starts from the root and builds everything with a mind for performance and modularity, but right now I haven't made any moves on that, and I don't think anyone else has either. It is probably preferable to work on a separate repo first, because any changes are likely to require modifications everywhere else in the ecosystem. AbstractMCMC makes it a lot easier to just hot-swap the chain backend.

@goedman
Copy link
Collaborator

goedman commented Jan 31, 2020

Thanks Cameron, looks like you and David are tackling the problem the right way, pretty much start from scratch with what you learned. I'll take a look at AbstractMCMC.

I've actually been playing with/thinking about another possible idea, derived from MonteCarloMeasurements, in being able to construct Particles (=sample chain) + Distribution + Distribution parameters (as identified by mcmc). But over the last 2 months clearly had other issues to look into. But I feel a lot better it has the right visibility now.

@devmotion
Copy link
Member

Seems the PR is actually fixing (parts of) the issue: https://discourse.julialang.org/t/slowdown-after-package-load/33533/13

@mileslucas
Copy link

Just wanted to toss in here that NestedSamplers.jl completely hangs when trying to use it or test it. I believe it is because of its dependence on MCMCChains.jl. I have commented out the import and Chains usage and I no longer hang. This is basically breaking that project since I can't locally test or even use it. I'm very interested in seeing any kind of fix for this.

@itsdfish
Copy link
Contributor Author

The only thing that I found to mitigate the problem in the short term is to restrict Requires to v0.5.2.

@mileslucas
Copy link

@itsdfish I am not using Requires at all, but still encounter this issue.

@itsdfish
Copy link
Contributor Author

Do you know if any of your dependencies are using Requires? You can check with ]st --manifest

@mileslucas
Copy link

mileslucas commented Feb 11, 2020

Yes, I'll try manually overriding it.

(NestedSamplers) pkg> st --manifest
Project NestedSamplers v0.1.0
    Status `~/dev/julia/NestedSamplers/Manifest.toml`
  [621f4979] AbstractFFTs v0.5.0
  [80f14c24] AbstractMCMC v0.3.0
  [7d9fca2a] Arpack v0.4.0
  [68821587] Arpack_jll v3.5.0+2
  [4fba245c] ArrayInterface v2.4.1
  [13072b0f] AxisAlgorithms v1.0.0
  [39de3d68] AxisArrays v0.4.2
  [b99e7846] BinaryProvider v0.5.8
  [324d7699] CategoricalArrays v0.7.7
  [aaaa29a8] Clustering v0.13.3
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v3.4.0
  [9a962f9c] DataAPI v1.1.0
  [a93c6f00] DataFrames v0.20.0
  [864edb3b] DataStructures v0.17.9
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.0.2
  [b552c78f] DiffRules v1.0.0
  [b4f34e82] Distances v0.8.2
  [31c24e10] Distributions v0.22.4
  [da5c29d0] EllipsisNotation v0.4.0
  [7a1cc6ca] FFTW v1.2.0
  [f5851436] FFTW_jll v3.3.9+3
  [1a297f60] FillArrays v0.8.4
  [6a86dc24] FiniteDiff v2.2.0
  [f6369f11] ForwardDiff v0.10.9
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+0
  [a98d9a8b] Interpolations v0.12.5
  [8197267c] IntervalSets v0.4.0
  [41ab1584] InvertedIndices v1.0.0
  [c8e1da08] IterTools v1.3.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [682c06a0] JSON v0.21.0
  [5ab0869b] KernelDensity v0.5.1
  [d3d80556] LineSearches v7.0.1
  [c7f686f2] MCMCChains v1.0.2
  [856f044c] MKL_jll v2019.0.117+2
  [e1d29d7a] Missings v0.4.3
  [d41bc354] NLSolversBase v7.6.1
  [77ba4419] NaNMath v0.3.3
  [b8a86587] NearestNeighbors v0.4.4
  [6fe1bfb0] OffsetArrays v1.0.2
  [4536629a] OpenBLAS_jll v0.3.7+5
  [efe28fd5] OpenSpecFun_jll v0.5.3+1
  [429524aa] Optim v0.20.1
  [bac558e1] OrderedCollections v1.1.0
  [90014a1f] PDMats v0.9.11
  [d96e819e] Parameters v0.12.0
  [69de0a69] Parsers v0.3.11
  [2dfb63ee] PooledArrays v0.5.3
  [85a6dd25] PositiveFactorizations v0.2.3
  [92933f4c] ProgressMeter v1.2.0
  [1fd47b50] QuadGK v2.3.1
  [b3c3ace0] RangeArrays v0.3.2
  [c84ed2f1] Ratios v0.4.0
  [3cdcf5f2] RecipesBase v0.7.0
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v1.0.1
  [79098fc4] Rmath v0.6.0
  [992d4aef] Showoff v0.3.1
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.9.0
  [90137ffa] StaticArrays v0.12.1
  [2913bbd2] StatsBase v0.32.0
  [4c63d2b9] StatsFuns v0.9.3
  [3783bdb8] TableTraits v1.0.0
  [bd369af6] Tables v0.2.11
  [efce3f68] WoodburyMatrices v0.5.0
  [2a0f44e3] Base64 
  [ade2ca70] Dates 
  [8bb1440f] DelimitedFiles 
  [8ba89e20] Distributed 
  [9fa8497b] Future 
  [b77e0a4c] InteractiveUtils 
  [76f85450] LibGit2 
  [8f399da3] Libdl 
  [37e2e46d] LinearAlgebra 
  [56ddb016] Logging 
  [d6f4376e] Markdown 
  [a63ad114] Mmap 
  [44cfe95a] Pkg 
  [de0858da] Printf 
  [3fa0cd96] REPL 
  [9a3f8284] Random 
  [ea8e919c] SHA 
  [9e88b42a] Serialization 
  [1a1011a3] SharedArrays 
  [6462fe0b] Sockets 
  [2f01184e] SparseArrays 
  [10745b16] Statistics 
  [4607b0f0] SuiteSparse 
  [8dfed614] Test 
  [cf7118a7] UUIDs 
  [4ec0a83e] Unicode

@itsdfish
Copy link
Contributor Author

If you are lucky, you can simply run ] add Requires @v0.5.2. But if that doesn't work, you might have to start a new environment and add Requires v0.5.2 first, followed by your other packages. Unfortunately, the versioning can be order-dependant. If I recall correctly, Julia 1.4 has a more permanent solution.

@mileslucas
Copy link

Yep, pinning Requires to 0.5.2 solved the problems I was having! Thanks!

@goedman
Copy link
Collaborator

goedman commented Feb 11, 2020

Hi Chris,

Does the pinning solve it or the fact that pre and final compilation work had been completed earlier?

I see similar issues with KernelDensity alone (which in turns polls in Optim), but only when I build a new version of J1.5-DEV. But on 1.3.1 and 1.4-RC1 the delays are far less pronounced after the very first time.

@itsdfish
Copy link
Contributor Author

As far as I can tell, pinning Requires v0.5.2 seems to solve it so long as your dependencies do not need Requires v1.0. My initial post shows the time differences on my local machine. There is a similar pattern on Travis, except MCMCChains with Requires v1.0 takes even longer, often causing a time out.

@devmotion
Copy link
Member

Is this still an issue with MCMCChains 3.0? At least for me the import times are drastically reduced (although it surely can be improved further).

@goedman
Copy link
Collaborator

goedman commented Feb 25, 2020

I just ran all StanModels tests (after rebuilding J1.5, so all packages will be recompiled) and no noticeable delay!

@cpfiffer
Copy link
Member

Awesome! Let's close this issue for now, but I think we've still got an eye on performance and load times moving forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants