Web compatibility and ESM in .js files #149
Description
This issue is a summary of a discussion starting at #142 (comment). There was a lot of debate on that thread; if you’d like to comment, please do so over there. The point of this issue is only to serve as a neutral document of the issue discussed in that thread. There’s also a separate thread for discussion of potential solutions, should the group decide to change anything.
The problem: ESM JavaScript in .js files
-
Node, in
--experimental-modules, always treats files with.jsextensions as CommonJS files. Within an ES module, animportstatement for a.jsfile will always try to parse that file as CommonJS; if the file is actually ESM—for example, it containsimportorexportstatements—the import will error. ESM files must be named with an.mjsextension to be importable via animportstatement in Node--experimental-modules. -
The Web always treats
.jsfiles as ESM. (This is an oversimplification, but the detailed explanation can be found in the next section.) Animportstatement of an URL that a webserver resolves to a.jsfile will be parsed by browsers as ESM. -
If the
.jsfile being imported is ambiguous, and could be parsed as either CommonJS or ESM, it will be executed differently in Node versus in browsers. Browsers will run it in strict mode, because all ESM is strict mode, while Node will run it in sloppy mode, the default for CommonJS. Since the exact same line of code to import this file can run in both environments—import './ambiguous.js'—this is a situation where identical code runs differently in Node as in browsers (as opposed to not running at all in one environment or the other). This is an incompatibility with the Web, and one of the primary goals of Node’s ES modules implementation is to achieve equivalence and compatibility with ES modules running in browsers. -
More generally, the Web allows
.jsfiles to containimportandexportsyntax, whereas--experimental-modulesdoes not. (Again, this is an oversimplification which will be explained below.) There are reasonable use cases where users targeting the Web may want (or even need) to save their ESM JavaScript in files with.jsextensions. If Node disallows ESM JavaScript from.jsfiles, this introduces another incompatibility with the Web.
But browsers don’t care about file extensions!
Indeed, they don’t. But they care about MIME types, and the simple version is that MIME types are to browsers (in ESM mode) as file extensions are to Node. When a browser evaluates import './file.js' in some ESM-mode JavaScript code, it asks a webserver for ./file.js (which is a relative URL, like what you see in <script type="module" src="./file.js">). The webserver resolves file.js however it deems fit to—maybe it finds a file named file.js, maybe it’s an API listening for a path named /file.js, maybe via some other method—and the server returns a string of JavaScript code and some headers. One of those headers must be Content-Type, and must contain one of the ESM-approved MIME types such as text/javascript or application/javascript. If it does, the browser will interpret that string as ESM-mode JavaScript and run it. If the MIME type is missing, or is a type that browsers don’t recognize (such as application/node, the MIME type for CommonJS) an error is thrown.
JavaScript is a static asset, so most of the time a webserver (or CDN or similar) will be serving it from a file. And the server needs to decide what MIME type to choose when it serves that file. Most, if not all, servers make that decision based on file extensions. And most will look at the .js extension and serve it as text/javascript, which browsers in ESM mode will treat as ESM. Webservers will also serve .mjs as text/javascript, which browsers will also treat as ESM. The .js and .mjs extensions are interchangeable as far as webservers and browsers are concerned. These file extension-to-MIME type mappings are usually configurable, though obviously not every user will have sufficient access to change these settings.
Node --experimental-modules as compared to browsers and webservers
Node choosing how to parse a file, for example deciding if a .js file should be treated as CommonJS or as ESM, is equivalent to the webserver deciding what MIME type to serve for the file: application/node (CommonJS) or text/javascript (ESM). Because no browser supports CommonJS, though, there are no known webservers in the world that serve .js as application/node. So webservers generally serve all .js files as text/javascript or application/javascript, which are interchangeable.
The complication for --experimental-modules is that webservers also serve .mjs as text/javascript. So both .js and .mjs are served with the same MIME type on the Web, and therefore webservers aren’t disambiguating anything beyond that the string to be served is some form of JavaScript. You can save an ESM JavaScript file with either a .js or an .mjs extension, and either will work just fine on the Web. Node’s --experimental-modules, on the other hand, doesn’t treat these identically—it loads .js as if it were a webserver serving it as application/node, a.k.a. CommonJS, so if a .js file contains import or export statements, that file will throw an error.
Just show me some code!
Here’s a demo you can run. The JavaScript code in that repo runs in both Node --experimental-modules and in browsers (as served by a typical webserver) and evaluates differently in each environment. The README shows the expected output, if you’d rather not clone the repo and run it yourself.
What about the spec?
The relevant specifications are silent on this matter. The import specifier (the string in import './file.js') can be anything per the JavaScript spec, and the browser spec currently requires it to be an absolute or relative URL. Node --experimental-modules essentially allows most of what require allows, and Node decides this on its own. There’s work to bring package imports, like import _ from 'lodash', to the Web via the package name maps proposal. There is no spec that governs what MIME type should be chosen for a file extension, nor could there by one, as JavaScript has several that are each valid depending on context.
It is this ambiguity that allows --experimental-modules to have its current behavior without violating any specs, though its behavior is clearly different than that of browsers and webservers.
So what should we do about this, if anything?
Because this isn’t meant to be a discussion thread, please use this thread to discuss potential solutions (or arguments for keeping the --experimental-modules current behavior).