[BUG] Installing a single git dependency takes 30 seconds

[Raynos]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When using npm 7 installing a single git dependency takes greater then 30 seconds

$ npm i optoolco/_base#be9c5f7 -S
npm http fetch GET 404 https://codeload.github.com/optoolco/_base/tar.gz/be9c5f727a4d94a95cee547d1cf761f193539806 235ms (cache skip)

changed 1 package in 37s

This is a private git dependency over git+ssh

For some more context, the repository is quite small

$ npm ls --prod --depth 999 | wc -l
75

$ npm ls --depth 999 | wc -l
100

I went into an empty directory and ran the same install command to see how long it takes there.

~/optoolco/foobar
$ npm i optoolco/_base#be9c5f7 -S
npm http fetch GET 200 https://registry.npmjs.org/ini 22ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@optoolco%2ftonic 25ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/node-fetch 23ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/tar 24ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@optoolco%2fcomponents 29ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/chownr 16ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/minipass 16ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp 14ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/minizlib 16ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/fs-minipass 18ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/yallist 15ms (cache hit)
npm http fetch GET 404 https://codeload.github.com/optoolco/opkit-node/tar.gz/a79014a348914c5a93a932d69a3609d3c60120d2 305ms (cache skip)
npm http fetch GET 404 https://codeload.github.com/optoolco/_base/tar.gz/be9c5f727a4d94a95cee547d1cf761f193539806 306ms (cache skip)

added 13 packages in 36s

Then I ran a similar command, to install a different version of the git dependency and received

$ npm i optoolco/_base#459cbb1 -S
npm http fetch GET 404 https://codeload.github.com/optoolco/_base/tar.gz/459cbb18b6d3382b7d84fb36a17cc64b8a5e35eb 386ms (cache skip)

changed 1 package in 37s

I'm suprised installing a single git dependency takes 37 seconds.

Expected Behavior

For an install of a single dependency to take less than 5 seconds

Steps To Reproduce

1. Install a git dependency
2. Wait a long time.

Environment

• OS: Linux
• Node: 14
• npm: 7.24.1

[Raynos]

Just to clarify, I am not installing a git repository that takes 30 seconds to clone

$ time git clone gh:optoolco/_base _base2
Cloning into '_base2'...
remote: Enumerating objects: 2007, done.
remote: Counting objects: 100% (2007/2007), done.
remote: Compressing objects: 100% (1307/1307), done.
remote: Total 2007 (delta 1034), reused 1585 (delta 622), pack-reused 0
Receiving objects: 100% (2007/2007), 1.13 MiB | 4.02 MiB/s, done.
Resolving deltas: 100% (1034/1034), done.

real	0m1.753s
user	0m0.158s
sys	0m0.050s

[Raynos]

I tried to reproduce this with a public github repository

$ npm i git+ssh://git@github.com/Raynos/npm-reproduction-base-example.git#62b6ea4 -S
npm http fetch GET 200 https://registry.npmjs.org/@optoolco%2ftonic 23ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@optoolco%2fcomponents 25ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/ini 26ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/node-fetch 25ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/tar 26ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/chownr 19ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/fs-minipass 18ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/yallist 15ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp 16ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/minizlib 18ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/minipass 19ms (cache hit)
npm http fetch GET 200 https://codeload.github.com/Raynos/tape-harness/tar.gz/27dae0e6678d2135478169783b85486133f60fd8 300ms (cache revalidated)
npm http fetch GET 200 https://codeload.github.com/Raynos/npm-reproduction-base-example/tar.gz/62b6ea430c7281d1cd52967df2e8a93b8eedb261 305ms (cache miss)

added 13 packages in 3s

I suspect the 200 vs 404 on codeload.github.com explains the difference between a 3s and a 37s install.

[Raynos]

I uploaded the npm timing information from npm install --verbose here https://gist.github.com/Raynos/c5defe132247cf98f59512983d30dddf

[ljharb]

What is codeload.github.com? Is it part of the dep graph of the git repo you’re installing?

[Raynos]

I have no idea what codeload.github.com is. I assumed it was something npm cli was using directly as an alternative to git clone.

Somewhere in the npm cli dependency tree it resolves something like substack/tape to codeload.github.com/substack/tape ?

[Raynos]

https://github.com/npm/hosted-git-info/blob/main/git-host-info.js#L28

I suspect it's coming from here.

[wraithgar]

Do you have two-factor auth or some other sort of prompt that happens when you do authenticated git operations?

[Raynos]

There is no two factor author or prompt.

It should be installing via git+ssh using my ssh credentials on my laptop. Looking at the timing output of ( https://gist.github.com/Raynos/c5defe132247cf98f59512983d30dddf ) my suspicion is that reify and idealTree are slow.

[BCsabaEngine]

Often I got 30-70sec reify with 7.x npm too. I try too analize it next time.

[kumarrishav]

facing same issue with github.com url as well

[luckydevil13]

+1 HERE