Add support for npm5

[JamieMason]

npm install -g npm5 has a different cache structure so shrinkpack will need updating to support that, while being backwards compatible for older versions.

[zkat]

So, there's two ways you can go about this: The way I think I most recommend is to run npm pack pkg@1.2.3 pkg2@3.4.5 other-pkg@1 ..., according to the versions in npm-shrinkwrap.json or package-lock.json. This way is compatible with just about any version of npm I know of, and does not even require a warm cache to work: it'll use the cache if needed, but will otherwise make sure whatever you need to fetch gets fetched before it's copied to the cwd.

The other way, which will be only compatible with npm@>=5, is to use cacache in conjunction with the content addresses in package-lock.json or npm-shrinkwrap.json. That is, cacache.get.stream.byDigest('~/.npm/_cacache', locked.dependencies[pkg].integrity).pipe(fs.createWriteStream(locked.dependencies[pkg].name + '.tgz'). Obvs with whatever other stuff you wanna add to this. Note that doing it this way is not all that different from going the npm pack route, except this version allows you to use/verify integrity hashes when available.

lmk if you have any questions about the cache format or the tools around it. shrinkpack is real cool. :)

[JamieMason]

Thanks @zkat, I'll most likely go the npm pack route 👍

[timdp]

In the meantime, it might be helpful to pass {stdio: ['pipe', 'pipe', 'inherit']} when you spawn npm so stderr doesn't get swallowed. Anyway, I appreciate the effort on this. 👍

[JamieMason]

Thanks a lot for the tip @timdp

[JamieMason]

@zkat should I consider rewriting the paths in package-lock.json in the same way I do npm-shrinkwrap.json (from URLs to their local .tar path) or would you advise against that?

[zkat]

@JamieMason If you do that, it HAS to be: file:relative/path/to.tar, and you have to add the sha512sum of the .tar to integrity (NOT replace it -- it needs to have both sha512s in that string, separated by a space. See https://npm.im/ssri)

I haven't tried this myself yet, but I would hope it would work. It should. If not, that's a bug I gotta fix.

So, to summarize:

"foo": {
  "version": "1.2.3",
  "resolved": "https://registry.npmjs.org/mypkg/-/mypkg-1.2.3.tgz",
  "integrity": "sha512-deadbeef"
}

becomes

"foo": {
  "version": "1.2.3",
  "resolved": "file:package.mirror/mypkg-1.2.3.tar",
  "integrity": "sha512-deadbeef sha512-c0ffee"
}

If the integrity hash doesn't match resolved, you'll get an integrity error.

[pietermees]

@JamieMason just wondering, any progress on this? Thanks!

[JamieMason]

Hey @zkat,
I've noticed a few mentions from npm peeps that there's a plan to ~implement shrinkpack in npm, is that right? If so, would there be any need to add npm v5 support to shrinkpack if it will be available without a 3rd party package anyway?

Thanks a lot,

Jamie

[zkat]

@JamieMason doing that is a ways away. You're free to implement it shrinkpack-side in he meantime -- it's gone pretty far down the queue in the past couple of months, and idk when we'd even get around to it, rn.

[JamieMason]

righto, thanks a lot @zkat

[cmandlbaur]

Is this something that is in progress?