Proposal: Alternative to enabling Corepack by default

[GeoffreyBooth]

This is a sketch of a proposal, so please don’t pepper me with questions about how it would work; I don’t know. It’s an early idea that I want to suggest to see if there’s interest.

So today, when you run corepack enable, a file named yarn is written to a path like /usr/local/bin/yarn. This is a #!/usr/bin/env node script that tells Corepack to run Yarn. Presumably #51886 will cause this file to exist upon Node installation, without the need to run corepack enable to create it. All well and good.

What if instead, we create a new #!/usr/bin/env node script that lives at the same path /usr/local/bin/yarn. This file would exist upon Node installation, and in this Node installation there’s no bundled Corepack (bear with me). When run, this new yarn script would do the following:

1. Prompt the user “Do you want to download and install Yarn?” and if yes:
2. npm install --global corepack
3. corepack enable yarn
4. Runs Yarn with the arguments originally passed to the script

From the user’s perspective, this is the same DX as if #51886 lands; though it takes slightly longer, because instead of downloading and installing just Yarn it’s downloading and installing both Corepack and Yarn.

Besides shrinking the Node bundle size slightly, the benefit of doing this is to get the Node project off the hook for the maintenance burden and security burden of Corepack. We would be responsible for this handful-of-lines download-and-install script, but that’s it. Corepack could be gifted to the Yarn org and they could become responsible for it.

This also becomes a pattern that we could use for any other CLI tools we may want to provide easy access to, but not bundle, in the future—even npm (downloaded and installed directly, not via Corepack). I don’t know what we might consider for such treatment other than Yarn and pnpm, but the capability would be there.

@nodejs/tsc @nodejs/corepack @nodejs/npm @nodejs/package-maintenance

[wesleytodd]

Just a first quick take on this, why npm install corepack and not npm install yarn? I agree with the goals of this and generally that this is a light improvement on the existing corepack, but I stand by the stuff corepack is providing is not necessary at all since we already ship a package manager.

[GeoffreyBooth]

Just a first quick take on this, why npm install corepack and not npm install yarn?

Because Yarn wishes to be installed via Corepack. npm install yarn installs Yarn 1.x.

I don’t know what pnpm’s preference would be. If they prefer direct npm install pnpm, that works. I think it would be up to whatever project we “bundle” this installer script for how they want the script to behave.

[wesleytodd]

Because Yarn wishes to be installed via Corepack.

The yarn package could do npx corepack enable yarn as a postinstall to setup the binary instead of relying on npm bin scripts? I think this is a problem which isn't a real problem for the node project to solve, but just presenting alternatives.

I think it would be up to whatever project we “bundle” this installer script for how they want the script to behave.

IMO this should be avoided. Since we still don't have clarity on what would then be included in this new shim thing and what the requirements would be to apply for inclusion I think it just brings back some of the problems corepack introduced.

[anonrig]

This proposal depends on npm to install corepack, and therefore requires npm to be existent. For future development, this feels like shooting our foot (one more time).

[wesleytodd]

This proposal depends on npm

I think that this proposal would assume something like this lands (or at least assumes the status quo): #51918

Also, node ships a package manager so I think it is expected that node uses that package manager to install packages. (I am only being as silly here as this conversation necessitates I swear).

[GeoffreyBooth]

If we remove npm, we just refactor this to download via fetch. Or it downloads via fetch in the first place. The implementation details are irrelevant.

The point is, is the use case of “As a user, I want to run Yarn without needing to install it first” one that we’re aiming to support? If so, this is a much lighter solution than shipping all of Corepack or all of Yarn.

[wraithgar]

we just refactor this to download via fetch

Unless the package manager in question bundles its dependencies, the only way for you to do this is to in fact implement your own package manager.

Package managers are a kind of unique snowflake because there has to be some point at which they can self-start. This is why npm has always bundled its own dependencies and why there is an install.sh file available via the npm website that lets folks install npm into a system with a missing or broken package manager.

Unless you're willing to solve that problem and take on the burden of maintaining your own package manager, or declaring bundled dependencies as a prerequisite to inclusion in this new process, it's going to be "turtles all the way down" as you will need a package manager to install your package manager.

npm install yarn installs Yarn 1.x.

This is because they have set 1.x as latest. The fact that corepack does something different breaks the convention of how the npm registry has always operated.

I was not around where and when corepack was being conceived or implemented, so I don't 100% understand what problems its trying to solve. It appears that there is a desire in at least part of the community to support having multiple versions of multiple package managers available at runtime. If that's the case perhaps that is an issue that can be solved specifically and intentionally, instead of having node get into the business of maintaining their own package manager. The current discussions around what is tentatively calling devEngines are promising in this regard.

[GeoffreyBooth]

I was not around where and when corepack was being conceived or implemented, so I don’t 100% understand what problems its trying to solve.

It was designed to solve the “problems from npm being the only package manager shipped by default” per https://github.com/nodejs/corepack/blob/967e2666b57f4ba33fb03397dbd6089251fc0503/DESIGN.md. A few days ago they deleted this design document without replacing it, and I’m unaware of any other documentation of Corepack’s intended goals or use cases, so it’s a bit unclear what problems Corepack is intending to solve. The TLDR is that it seemed to be originally intended as a way to remove npm from Node’s distribution, but over the years it has found its most usage as a version manager for pnpm to prevent pnpm from corrupting its own lockfile.

This proposal isn’t intended to be a way to unbundle npm, as I don’t think that that’s a goal that the project wishes to pursue at this time; however I think that if we change our minds on that, then any solution would involve the general approach described here. Yes it would presumably be complicated, and maybe use install.sh or something similar, but those are details we don’t need to work out unless and until we decide that removing npm is something we want to achieve. I don’t think any unbundling of npm from Node would involve Corepack, as the npm team is adamant that they don’t want npm installed by Corepack; so we could take the first step of adding these Yarn and pnpm installer scripts to unbundle Corepack and leave the question of npm for another day.

[wraithgar]

Thanks @GeoffreyBooth, this is not an easy conversation to be late into joining. I appreciate your help in fielding my basic questions.

I don't have much to add then at this point. If there are specific questions you feel npm can answer please ping me, or bring it up over in the package metadata interop repo (if applicable)

[aduh95]

declaring bundled dependencies as a prerequisite to inclusion in this new process

Bundled dependencies are a prerequisite for working with Corepack, so it wouldn't be too bad to have the same requirement.

[styfle]

This proposal depends on npm to install corepack, and therefore requires npm to be existent. For future development, this feels like shooting our foot (one more time).

Agreed. Something doesn't seem right.

The new script could potentially fetch and untar from the registry without invoking npm, but then you've basically re-implemented what corepack already does.