When to make `--default-type=module` the Node.js default

[GeoffreyBooth]

Now that nodejs/node#49869 has landed, and we need to figure out what to say about it for the 21.0.0 announcement, we should agree on a tentative plan for when the default value of --default-type flips from commonjs to module. That would be a semver-major change.

Obviously we can’t announce a certain date or version, since we don’t know what issues may arise during 21.x as people test out --experimental-default-type=module and when we might consider it stable, but assuming things go fairly smoothly then we could conceivably make the flip in 22.0.0 in April 2024. I think that would probably be preferable to 23.0.0 in October 2024, which would mean that the first LTS line with ESM by default wouldn’t begin until 24.x in April 2025, which is a long way off.

I think for the health of the project and the greater JavaScript ecosystem it’s important for Node.js to shift to an ESM-first mindset. We will continue to support CommonJS as a first-class module system, but it becomes opt-in rather than opt-out, conforming Node with other runtimes and modern tools.

This is primarily a change for new users. Most users learning JavaScript today are learning it with import/export syntax and get frustrated when Node doesn’t run their code right away without messing with configuration settings. I think it’s important that we prioritize the UX of these users. The users still writing CommonJS today are among our most sophisticated and experienced, and will be able to figure out how to opt into the old mode via --default-type or the "type" field.

[GeoffreyBooth]

For those wondering, this is what the migration paths look like:

• For users already writing ESM, either via package.json "type": "module" or .mjs, nothing needs to be done.

• For users writing CommonJS apps or libraries, where their code is in a folder that has a package.json file, they would need to add "type": "commonjs" to that package.json file if they haven’t already; or run Node with --default-type=commonjs.

• For users writing “loose” .js or extensionless JavaScript files, that have no package.json file in the folder alongside them or in any parent folder, the user would need to do one of the following:

  • Add a package.json file with "type": "commonjs".
  • Rename the file to end in .cjs.
  • Make the path of the file into a symlink to a file that either ends in .cjs or is under a folder with a package.json file with "type": "commonjs".
  • If the file is an executable script with a shebang line and their system supports env with the -S option, change the shebang to #!/usr/bin/env -S node --default-type=commonjs.
  • Use some shell scripting to have the script passed to Node via STDIN with --input-type=commonjs.

Or, of course, they could refactor code from CommonJS to ESM, but I’m not sure that needs to be presented as an option. This is nothing like Python 2 to 3.

The other significant migration is of packages that rely on monkey-patching the CommonJS loader. The authors of major instrumentation libraries and of Yarn Plug-n-Play have been very active engaging with the loaders team in ensuring that the module customization hooks can be used to support their use cases. Those APIs are already in “release candidate” stage, just baking to see if anyone reports any bugs before we declare them stable. Regardless of the --default-type flip, though, it’s important for the maintainability of the modules subsystem that anyone who currently monkey-patches moves away from that pattern and onto public APIs. The need to support monkey-patching use cases has severely hampered our ability to do significant refactoring or improvements of the modules code. Substantial performance achievements, such as could be attained by moving parts of the modules code into C++, can only be done if we can break the already unsupported monkey-patching use cases. Having an official announcement that the ESM loader will become the default module loader in a future Node version will hopefully be what corporate teams need to prioritize the work to move away from monkey-patching.

[ljharb]

Doesn’t this mean that users writing a library would have different default behavior for themselves than for when they’re installed in someone else’s node_modules?

[mcollina]

Doesn’t this mean that users writing a library would have different default behavior for themselves than for when they’re installed in someone else’s node_modules?

Yes. I think changing that is going to a be a massive breaking change that would break everybody. Most modules would need to keep working with the flag.

Other alternatives are

1. deprecate typeless packages outside of node_modules
2. ask npm init and the like to include a type property by default

I would personally prefer to ship this in 23, having v24 the first LTS with the change, but I'm not opposed to an accelerated timeline.

If we decide not to flip the switch in v22, we could ship some form of warning in v22 for typeless packages outside of node_modules, taking a similar "iteration" approach that we used in the past.

Something to consider is the coordination with the TS team, as this will impact them and their resolution algorithms.

Of course, Loaders should be stable by then and no critical bugs should be there (or in ESM) to do the switch.

[GeoffreyBooth]

Yes. I think changing that is going to a be a massive breaking change that would break everybody. Most modules would need to keep working with the flag.

Let’s investigate this, shall we? Let’s go through some scenarios:

1. ESM-only packages, like node-fetch: The flip has no effect, on either the package author or its consumers.

2. CommonJS-only packages, like express, where the author is developing the package using a pre-flip version of Node and the consumer is running post-flip Node: The flip has no effect, on either the package author or its consumers.

3. CommonJS-only packages, like express, where the author upgrades to the post-flip version of Node. Assume that the package uses .js extensions and the package.json file has no type field: The first time the author tries to develop their package after upgrading, it will fail to load because .js is ESM now. The error message will prompt the author to add "type": "commonjs" to their package.json. The author does so and is back to developing. No effect on consumers of the package.

4. A new ESM-only package, where the author starts writing the library in .js files with a package.json file that has no "type" field: (Assume for the sake of evaluating worst case scenarios that we get no help from npm or build tools; they function as they do today.) The author publishes their package to npm, thinking all is fine, but any consumers of the package—both using pre- and post-flip versions of Node—see errors when they try to consume it, because in all versions of Node a type-less package under node_modules is interpreted as CommonJS. This published version of the package is effectively broken for all users, which would surely trickle back to the author, who would add "type": "module" to the package.json and publish a new version that would work for everyone. Because this is a new package, it wouldn’t have millions of consumers; the mistake would be rectified very early on, before it has a chance to grow its popularity.

So basically, package authors would need to learn to add the type field to their package.json files, and hopefully we get some help from npm or other tools reminding them to do so. But the consequences of them forgetting to do so are low: a package either works for all users or is broken for all users, which is really what we want, because broken versions won’t find any adoption. I don’t see any scenarios where either authors or consumers would use --default-type=commonjs; it’s easier and more permanent to just add "type": "commonjs". Are there any other plausible scenarios I haven’t thought of?

[benjamingr]

Excellent summary @GeoffreyBooth that really clarified how migration would look and it sounds very reasonable.

We should also contact npm and ask for:

• Adding "type": "module" by default to npm init (as suggested by Matteo)
• Check that type is defined in the package.json during npm publish

@nodejs/npm does that sound reasonable?

Edit: opened an issue with npm: npm/cli#6855

[GeoffreyBooth]

We should also contact npm

For reference, adding type to npm init was proposed and rejected in 2019, then proposed and rejected again in 2021, then proposed again in 2022 (still open) and again in 2023. Of course, all of those prior discussions happened before the proposal of Node changing its default, so I am optimistic that a new outcome might be possible once we make an announcement on our side.

[wraithgar]

Am I following this correctly, that every package that's currently published will stop working once this goes into effect, and only newer versions of those packages that have this field defined will work?

[benjamingr]

@wraithgar no, the issue is with the default changing outside node_modules, the issue is when people publish new versions of packages without specifying type and relying on the default being esm - which is why we want the small npm enhancement to let people know their published code requires "type" to work as it did before publishing.

[joyeecheung]

I think this means if you have a simple script like this:

// test.js
require('fs')

This would stop working once the flag is flipped, unless you change the file name or add a package.json. I believe there is an enormous amount of utility scripts and tutorial examples that would be affected, now the question is do we want to create this havoc in all those scripts already out there and create a python2-like situation...

[joyeecheung]

Also, wouldn't this break most of our tests unless we do something? (and to that extent, it could probably break a lot of other tests in other projects that are run in a similar "loose" fashion). It could also force all projects not yet migrated to do something, while we already know that there are a lot of simple projects that are not actively maintained but are still heavily depended upon in the ecosystem - or the Node.js ecosystem once favored very simple packages like this, so if an author has tons of very simple packages that do not need much maintenance since Node.js v0.x days (because who would've thought that Node.js is going to break such simple use cases), they will now be forced to update all these small packages. And many of the old reproductions of confirmed bugs would also need updating...

[GeoffreyBooth]

Also, wouldn’t this break most of our tests unless we do something? (and to that extent, it could probably break a lot of other tests in other projects that are run in a similar “loose” fashion). It could also force all projects not yet migrated to do something,

The “do something” is what I wrote above as the migration path for CommonJS projects with no type field: add "type": "commonjs" to package.json, or create a package.json containing only { "type": "commonjs" }. That’s it.

People seem to keep pointing at Python 2-3 as the only example of a migration, but this is nothing like that: that required everyone to refactor their code, whereas this requires no one to refactor their code. They can if they want to, but this is a configuration change not a syntax change.

Yes the “shell scripts” case is the most awkward, because people often lack package.json files for those. But as I’ve written elsewhere, those scripts aren’t our primary use case and they shouldn’t be prioritized over application developers. Per https://nodejs.org/en/about, “Node.js is designed to build scalable network applications.” Just about every application framework has instructions in ESM syntax. TypeScript has exclusively supported ESM syntax since its inception, if I’m remembering correctly. We’re more out of sync with the ecosystem by requiring ESM to be opt-in than opt-out; while there will be some old tutorials and examples that become outdated after this change, there is far more that’s the reverse. Having that code work by default in Node.js will improve the new user experience and bring us into conformity with the broader ecosystem.