Pipenv very slow. Takes an hour to install and lock.

[ScarletMcLearn]

Pipenv is very slow.

It takes a long time to download packages.

Install packages.

And lock file.

[Jamim]

Unfortunately, this is true.

By the way, do you use some additional indexes (sources) in your Pipfile?
If so, that is the possible reason of the issue.
#2730 (comment)

[devkral]

The "Poetry" (package manager) project has much faster hashing.
It would be cool if you would use their algorithm (or faster).

[devkral]

And I don't get why something simple like --help needs 5 seconds.
Maybe some code pathes should be lazy loaded.

[techalchemy]

We don’t use a special algorithm for hashing, we use sha256. I suspect poetry just trusts the hashes it’s told if it’s significantly faster at hashing. We re-hash every file for security.

If it takes 5 seconds to run simple arguments that’s an issue and you should open an issue about it with details. We’d be glad to merge anything that gets us performance gains without costs to functionality

[devkral]

Trusting the hashes? Sounds like a good idea if the hashes are checked when using the lock-file.
Pip checks the hashes too, doesn't it?
Anyway: do you request the packages sequentially? I suspect some loops, serial lookup and extensive object generation.
I think the slow help call is also a symptom of this.
When I scanned skimmed the code I saw that the vendor package is not a namespace package. Every request to vendor loads all packages in it => namespaces are lazy.

[devkral]

Even worse: patched standard packages. And patched is also no namespace package.
def _patch_path(): loads all vendored packages. Again performance!
Use a lazy loader which patches and caches on the fly.

[devkral]

Even worse: patched standard packages. And patched is like vendor: no namespace package. def _patch_path(): loads all vendored packages. Again performance! Use a lazy loader which patches and caches on the fly (if you really want to patch). Pip has some options to specify cache directories, use them instead patching. In my opinion pipenv should also not use some fancy stuff like click even colors are gone this way. It is just bloat for a core component. I could not even pinpoint the slowdown because of so many dependencies (conditional loading and capsulation could help). I get slowly the gut feeling that a re-implementation would be easier than fixing this project. That is what makes poetry fast. Sry for the rant. I would really like to use pipenv (and used it in the past) but it is too slow. When I had no good internet, poetry worked within minutes while pipenv took several hours.

[rooterkyberian]

@devkral
using time: pipenv --help 0,45s user 0,07s system 99% cpu 0,522 total , so it is noticeable, but seeing how "rare" its to invoke pipenv without some even more time-consuming action (like installing packages) IMO its not a really a problem. Lazy loading would be nice, of course, but more benefit could be found in other places.

What I personally oppose is the notion to remove eye candies, that actually speed up lookup of information, just to get some milliseconds back of CPU time at the cost of milliseconds to seconds of human time (be it during use, or development/support time).

[bhooshan-gadre]

I tried it today for the first time. But it is taking more than 30 mins now to install ipython and jupyter:
after creating an env using pipenv using python 2.7 and then doing pipenv shell, I thried following
pipenv install ipython[all] jupyterlab as it is still Pinning VCS Packages... which is just one of my python git repos.

[ekhaydarov]

even though there hasnt been new releases in a while ive noticed a marked slowdown in pipenv update mainly because the travis ci build times out after 10m of inactivity. Nothing new in terms of extra packages.

[Peque]

@techalchemy Duplicate of #2284 ?

[frostming]

Close for duplcating