Define spec for development environment field

[GeoffreyBooth]

This is a draft spec for nodejs/node#51888 (comment), to define the runtime and package manager for developing a project or package (not consuming/installing it as a dependency in another project). We could use the name devEngines and it would be a new top-level field defined with this schema:

interface DevEngines {
  os?: DevEngineDependency | DevEngineDependency[];
  cpu?: DevEngineDependency | DevEngineDependency[];
  libc?: DevEngineDependency | DevEngineDependency[];
  runtime?: DevEngineDependency | DevEngineDependency[];
  packageManager?: DevEngineDependency | DevEngineDependency[];
}

interface DevEngineDependency {
  name: string;
  version?: string;
  onFail?: 'ignore' | 'warn' | 'error' | 'download';
}

The os, cpu, libc, runtime, and packageManager sub-fields could either be an object or an array of objects, if the user wants to define multiple acceptable OSes, CPUs, C compilers, runtimes or package managers. The first acceptable option would be used, and onFail would be triggered for the last defined option if none validate. If unspecified, onFail defaults to error for the non-array notation; or it defaults to error for the last element in the array and ignore for all prior elements, for the array notation. Validation would check the name and version ranges.

The name field would be a string, corresponding to different sources depending on the parent field:

• os: NPM docs for engines.os.
• cpu: NPM docs for engines.cpu.
• libc: glibc or musl; see NPM docs for config.libc.
• runtime: WinterCG Runtime Keys.
• packageManager . . . I don’t know, we could define the list as part of the spec, or perhaps it would need to correspond to an npm registry package name. Suggestions welcome.

The version field syntax would match that defined for engines.node, so something like ">= 16.0.0 < 22" or ">= 20". If unspecified, any version matches.

The onFail field defines what should happen if validation fails:

• ignore: nothing.
• warn: print something and continue.
• error: print something and exit.
• download: remediate the validation failure by downloading the requested tool/version.

In the event of onFail: 'download', it would be the responsibility of the tool to determine what and how to download, perhaps by looking in the tool’s associated lockfile for a specific version and integrity hash. It could also be supported on a case-by-case basis, like perhaps Yarn and pnpm could support downloading a satisfactory version while npm would error.

Typical example:

"devEngines": {
  "runtime": {
    "name": "node",
    "version": ">= 20.0.0",
    "onFail": "error"
  },
  "packageManager": {
    "name": "yarn",
    "version": "3.2.3",
    "onFail": "download"
  }
}

“Uses every possible field” example:

"devEngines": {
  "os": {
    "name": "darwin",
    "version": ">= 23.0.0"
  },
  "cpu": [
    {
      "name": "arm"
    }, {
      "name": "x86"
    }
  ],
  "libc": {
    "name": "glibc"
  },
  "runtime": [
    {
      "name": "bun",
      "version": ">= 1.0.0",
      "onFail": "ignore"
    },
    {
      "name": "node",
      "version": ">= 20.0.0",
      "onFail": "error"
    },
  ],
  "packageManager": [
    {
      "name": "bun",
      "version": ">= 1.0.0",
      "onFail": "ignore"
    },
    {
      "name": "yarn",
      "version": "3.2.3",
      "onFail": "download"
    }
  ]
}

Some potential future expansions of this spec that have been discussed are:

• runtime and packageManager might take shorthand string values defining the desired name or name with version/version range.

[ljharb]

Can you just comment the proposal here? This repo only exists to document what already exists; it doesn't make sense to make a PR until that's the case.

[wesleytodd]

I agree and think we should write this proposal up as prose first. I think we set the initial scope to be "documentation only" as a way to take off a good split task that was not huge scope. Technically this repo is not even for that documentation, this one is: https://github.com/openjs-foundation/package-json-research

[GeoffreyBooth]

Okay, how about something like this. We could use the name devEngines and it would be defined with this schema:

interface DevEngines {
  runtime?: DevEngineDependency | DevEngineDependency[];
  packageManager?: DevEngineDependency | DevEngineDependency[];
}

interface DevEngineDependency {
  name: string;
  version?: string;
  os?: string[];
  cpu?: string[];
  onFail?: 'ignore' | 'warn' | 'error' | 'download';
  download?: {
    url: string;
    hash: string;
  }
}

Both runtime and packageManager sub-fields could either be an object or an array of objects, if the user wants to define multiple acceptable runtimes or package managers. The first acceptable option would be used, and onFail would be triggered for the last defined option if none validate. Validation would check the version ranges, OS and CPU constraints.

The runtime.name field would be a string from the list of WinterCG Runtime Keys. The packageManager.name field . . . I don’t know, we could define the list as part of the spec, or perhaps it would need to correspond to an npm registry package name. Suggestions welcome.

The version field would match the definition of the value of engines.node, so something like ">= 16.0.0 < 22" or ">= 20". If unspecified, any version matches.

The os and cpu fields would match their correspondents in https://docs.npmjs.com/cli/v10/configuring-npm/package-json#os and https://docs.npmjs.com/cli/v10/configuring-npm/package-json#cpu. If unspecified, any OS or CPU matches.

The download field would apply only for onFail: 'download'. I assume no one would support for runtime. It could also be supported on a case-by-case basis, like perhaps Yarn and pnpm could support downloading a satisfactory version while npm would error.

I think this covers all the ideas from nodejs/node#51888 (comment) and nodejs/node#51888 (comment). cc @wraithgar

[wraithgar]

onFail is a user directive, not a maintainer directive. It would be unwise to let the maintainers dictate that behavior. A good historic example of why this is not advisable is engines itself. There are many packages out there that specify an engines that looks something like node: "^8.0.0". The maintainer likely had every intention of keeping that up to date but now that package doesn't work in node 20. If they also had been able to specify "fail" then the package would stop working, even if it works fine. Because packages are idempotent this can't be fixed. The end user is ultimately the only one who can make the decision on whether or not this constraint is a blocker.

[GeoffreyBooth]

onFail is a user directive, not a maintainer directive.

But in this case, the user is the maintainer. This whole configuration block only applies when it’s the developer of the package/project.

[wraithgar]

How does download even work with a semver range? If these are all npm registry packages then there is already a well defined discovery mechanism for downloading a given named package at a given version.

I'd love to hear from yarn or pnpm if and how they'd plan on handling the failure state where the user has indicated they want to download the correct version. How do you pick the version? Do you spawn it yourself? Why not fail and let the user remediate?

[wraithgar]

But in this case, the user is the maintainer

Good point. As you can see the lines between "installing as module" and "interacting as app" are so blurry that even I missed that.

[GeoffreyBooth]

How does download even work with a semver range?

If a URL is provided, we’re just downloading that exact URL. The version constraint wouldn’t apply.

If we want to allow onFail: 'download' and a missing download field, the only behavior I can think of would be to query the npm registry for the requested package manager and download the newest version that fits within the range.

[wesleytodd]

Yeah to be clear all of this is for the maintainer. We should include that this proposal would not effect any install behavior other than for the top level project.

[GeoffreyBooth]

I moved my draft into the top post to make it easier to find.

[ljharb]

This is a great start. I think it's important that it not just be limited to the runtime and the package manager - process.versions has a lot of things in it that would be really nice to also constrain in various cases, especially openssl which has lots of CVEs.

[styfle]

Tagging relevant people to review the draft spec: @Ethan-Arrowood @arcanis @zkochan @aduh95

[wraithgar]

I am not very good at reading specs like this. What would it look like in implementation to limit the local dev environment to a given os or cpu for example? From what I see it doesn't look like that field is sufficiently decoupled from "named" which doesn't make sense in that context.