Include SemVer in Node.js

[bnb]

It would (imo) be valid to say that SemVer a fundamental and essential part of the ecosystem. All modules rely on it, Node.js itself uses it for versioning, and it's currently sitting at 193m downloads last week, and 187m downloads the week prior - with a growth of 6m downloads in a week.

During both of the tooling sessions at the collaborator's summit, we discussed vendoring semver. There seemed to be general consensus, but I wanted to create an issue to begin the discussion.

Discussion points:

• Should we do this?
  • General consensus from the collaborator summit seems to be that this is an acceptable addition.
• Would the semver repo be able to be moved into the Node.js org?
  • There seems to be a willingness/interest in this from the current maintainers.
• How does this get included?
  • There was a general consensus / a lot of discussion around having a vendored scope. This does seem to be an important discussion that should be explored and expanded separately and simultaneously.

I'm sure there's a few more points here, but wanted to get this down so we can begin to have it.

[ruyadorno]

To add to the vendor scope/idea: Since the original conversation about including semver into Node.js during the collaborator summit, a thread was spawned in the node repo (ref: nodejs/node#43413) about the idea of vendoring dependencies but I personally find that conversation got derailed due to the suggestion of opening the door to potentially adding any arbitrary package from the ecosystem, I believe if we're to start again that conversation it should be under the prism of only vendoring libraries that are already under the Node.js project (such as undici) and only consider including semver once it gets transferred over to Node.js.

[ruyadorno]

A different idea from these originally pointed out in this thread that @darcyclarke brought up in today's meeting is to potentially try to see if there's interest in adding some level of semver support to the language specification itself, potentially spawning a TC39 study group (not sure on this terminology) as a better place to collaborate.

[ljharb]

I suspect that would be a tough sell unless there’s very compelling universal (browser, IoT) use cases I’m unaware of.

[lirantal]

Joining @ruyadorno's concern, if I understood correctly, that vendoring ecosystem dependencies would easily open the door to malicious packages in a nested tree to slip in. However, there's value in having common capabilities like semver vendored in and be semi-part of the runtime if so to say and I am wondering if the following idea might make a good compromise - both undici and semver are 0 dependencies (semver technically has 1 but authored by same person), and both are part of trusted individuals in the ecosystem. And so with that intro, could we make the case that whichever dependencies we vendor have to uphold to such specific thresholds? for example:

• They're an OpenJS Foundation project
• They're maintained by trusted individuals (e.g. that person is part of the Node.js org)
• They have virtually 0 dependencies

[styfle]

semver technically has 1 [dependency] but authored by same person

Would that dependency (lru-cache) also be exposed or just semver?

[ruyadorno]

Would that dependency (lru-cache) also be exposed or just semver?

Taking a quick look at the code it seems that lru-cache is only used in a single place in order to memoize a hot path, with that in mind it doesn't sound unfeasible to replace that with any memoize/cache implementation that core is already using or alternatively, look into also transferring that package over to the nodejs org.

That's a good point @styfle, this is an item that would need to be taken into consideration if we're to include semver in node core, regardless of how it happens (vendor, util, etc).

[darcyclarke]

The quick facts/data on node-semver's adoption:

Compared to the Ecosystem

ref. https://npmtrends.com/lodash-vs-react-vs-semver

Compared to the npm CLI Team's Supported Projects

As shown above, node-semver represents a disproportionate amount of the installs we see & is only growing. When compared to other popular packages (ex. react or lodash) the package dependants to install ratio is extremely low which is a good indicator that the usage is being driven by end-users/consumers (not maintainers).

Name	Dependants	Downloads (weekly)
semver	~22k	~199.5 million
lodash	~120k	~45.5 million
react	~90k	~15.5 million

I know I was the one to originally suggest potentially vendoring &/or including the capabilities of node-semver into core (at OpenJS World back in June) but I've taken a step back since & realized this likely needs to be a bigger initiative (I also just am not a fan of the vendoring concept in-general, but appreciate the work @jasnell & others did to investigate that fully). I'm proposing we bring SemVer & the capabilities of node-semver module to TC39 &/or W3C.

Beyond the CLI & cache dependency (which can be easily bundled or removed), there's nothing really unique or overtly tied to Node about node-semver.

Unfortunately, there are nuances to node-semver which deviate from the canonical SemVer "standard" (which is why you'll see many folks, including myself reference the project as "node-semver" even though the package name its distributed under is "semver").

The SemVer org, unfortunately, has stagnated & not kept pace with the innovations in package/version management. Other language ecosystems (ie. Ruby, Python etc.) have also chosen to deviate from the "standard" which has created a fragmented understanding of what is & isn't valid in regards to SemVer Ranges & Comparators. That said, these differences should not prevent us from trying to unify the understanding within the JavaScript community itself & we should scope the unification effort to this language to avoid undue scope creep.

As an easy first step, we can & will abstract the core library to be independent of dependencies & runtime-agnostic; immediately allowing other ecosystems to share our understanding of versioning (ex. bun, deno & the front-end). Notably, there is prior art here with somewhat significant adoption (ref. https://www.npmjs.com/package/@storybook/semver & https://www.npmjs.com/package/compare-versions).

A rough breakdown of how I imagine the next 12-24 months in regards to this overarching initiative:

• ~1-2mo The npm CLI team will ship a new major version of semver (ie. semver@8)
  • pulling out the CLI into a separate library
  • refactoring the code to make it runtime agnostic
  • mitigating most any/all API breaking changes to the core functions
• ~6mo An initial Stage 0 TC39 Proposal &/or a W3C Web API Proposal is created by {{insert willing participants}} that includes definitions which outline the existing capabilities of semver along with the various usecases/examples this core understanding/API will unlock for the ecosystem at-large
• ~1yr Have a new WG spun up within TC39 &/or W3C dedicated to formalizing the proposal/spec (will become the champions to move it through the next stages)
• ~1-2yr Land support for semver capabilities into the core language (ie. ECMAScript) or standardized as a Web API

This is obviously ambitious & I'm certain things will change along the way here but hopefully this initial outline gets some folks interested/excited.

[ljharb]

I don’t see how it would fit with W3C, since it has zero correlation with the web - as for TC39, I’d expect a lot of pushback for something node-specific (since browsers wouldn’t likely want to implement something that has minimal use on the web).

I agree that vendoring is not an ideal solution, but i don’t understand why node, and only node, isn’t the place to house this package and/or implementation.

[bakkot]

I would put it a slightly different way: in my estimation, browsers are very unlikely to be interested in shipping semver to their users. So even if W3C or TC39 were interested in standardizing it, it would be dead letter. I could be wrong about this, but you should probably check before going any further down this path.

I do think there's value in having this API widely available and standardized across server-side runtimes, like node/deno/bun - especially if there's buy-in from other languages and ecosystems - but the mentioned standards bodies are unlikely to want to be responsible for a standard which is not particularly useful in browsers. (Unless there is some reasonably common use case in browsers I'm missing?)

This perhaps speaks to the need for a different body where server-side APIs like this could be standardized. Or you could just write up a spec and conformance tests and see if you can get Deno and Bun to buy into it, rather than trying to spin up an actual standards org.

[styfle]

Probably not W3C or TC39 but perhaps https://wintercg.org would be interested in standardizing

[jasnell]

It's something that can be discussed but unless there's an API proposal to standardize around that is implemented by multiple runtimes it's going to be difficult to make progress