Pull request opened for import.meta.require on core

[MylesBorins]

Hey all,

I've opened a PR to introduce import.meta.require to core

nodejs/node#21317

I think this feature is needed no matter which overall implementation / feature set we move forward with. I'm adding this to the modules agenda and would like to see us reach consesnsus on landing this if possible.

If individuals have objections or concerns please let me know, either in this thread, the PR, or privately.

[MylesBorins]

@guybedford had voiced an objection to the feature in the pull request

I'm -1 on this for two reasons:

1. The way that a module is imported has to change depending on the module format. This puts cognitive overhead on the user above just "import a module".
2. Code analysis, transpilers, and bundlers will need to adapt to this. For example Rollup can do module analysis due to the bindings nature of ES modules. If import.meta.require is required to be supported in Rollup (which would be the case for CommonJS modules being imported this way in the Node ecosystem), then this could interfere with the current treeshaking benefits Rollup provides.

---

My response

@guybedford to your two points

The way that a module is imported has to change depending on the module format. This puts cognitive overhead on the user above just "import a module".

There will be a follow up PR to this in which I plan to propose removing transparent interoperability. It seems to me like this specific point only becomes an issue if we remove that capability. In the mean time, with the current implementation, individuals are free to statically import a cjs module or dynamically import it. If people are planning to dynamically bring in CJS it does seems odd to require them to wrap it in a process via dynamic import.

When I propose removing transparent interoperability I plan to implement a way to trap errors related to importing common.js and give extremely clear errors about how to use import.meta.require. As it stands right now people already need to know the "type" of the module as CJS cannot do named exports. I personally ran into this exact problem trying to import our test common module. The current behavior is not very clear and one has to know not only the type of module, but the specific edge case to our implementation as well as the fact that there is a difference between named exports and destructuring. I do not consider that simpler.

Code analysis, transpilers, and bundlers will need to adapt to this. For example Rollup can do module analysis due to the bindings nature of ES modules. If import.meta.require is required to be supported in Rollup (which would be the case for CommonJS modules being imported this way in the Node ecosystem), then this could interfere with the current treeshaking benefits Rollup provides.

While I will agree that this is a potential edge case I am not convinced that this specific case significantly breaks expectations. People are already using both import and require liberally in their code being transpiled and bundled by tools like rollup and webpack. These tools already need to recognize the require keyword, and it would seem that identifying import.meta.require would be fairly similar. I'd like to here from @Rich-Harris and @TheLarkInn about their thoughts. I for one think that being explicit in these cases, and removing the ambiguity around import (if we eventually remove transparent interop) would eventually simpify tree shaking algorithms. I am very open to being mistaken on this.

Moving conversation here so we can focus on technical implementation in the PR

[justinfagnani]

+1 to removing transparent interop. IMO, removing it will greatly simplify things. I don't think very many modules will transparently upgrade from CJS to JS modules - there's almost certainly going to be breaking API changes around exports.

Regarding @guybedford's later comment:

Another important point here is that here is no way for browsers to provide import.meta.require. One could imagine import react From 'react' being pointed to a CDN source on the web, and from CommonJS in Node, but such a mapping system isn't possible with import.meta.require - the code would not work in the browser without a build step that involves altering the code of the module.

With transparent interop on the other hand, something as simple as a package name map could provide this support.

Modules targeting browsers should not ever use import.meta.require as there's no way for browsers to implement require()'s synchronous semantics anyway. It would only ever be a feature of bundlers. Package name maps would only let code resolve a name to a URL, not fetch an asset and it's transitive dependencies synchronously.

[ljharb]

@justinfagnani that's off topic for this issue; let's keep this thread strictly to import.meta.require.

[guybedford]

@ljharb this thread does directly relate to transparent introp as the alternative mechanism for importing CommonJS, to restrict those arguments is to ignore key points.

[ljharb]

@guybedford i agree, but discussing removal of transparent interop is a highly contentious question that will be unlikely to ever attain consensus, and conducting that here will derail the central question of this issue.

[mAAdhaTTah]

One could imagine import react From 'react' being pointed to a CDN source on the web, and from CommonJS in Node ... With transparent interop on the other hand, something as simple as a package name map could provide this support.

Are you likely to end up in this situation. If React provides an ESM build, why not use the ESM build in Node as well? If a package provides both ESM & CJS, and you're using ESM in your app, would you need to point at the CJS version at all? In this formulation, import.meta.require is useful primarily for importing packages that only have a CJS build, which is going to require a build step anyway.

[bmeck]

I think the ability to import CJS has a very important note here. import.meta.require does not satisfy all the use cases of being able to import CJS. I don't think we should associate this PR as a means to remove importing CJS. In particular the ability to import CJS allows for side effect / singleton dependencies to interleave with ESM dependencies. e.g.

import 'cjs';
import 'esm';

vs

import.meta.require('cjs');
import 'esm';

This means that if you have a dependency on a module that is CJS being evaluated in a specific order in your module graph (generally before some dependency in order to configure a singleton / configuration / polyfill / etc.), that import.meta.require cannot fulfill that use case. There are a variety of workarounds if you start using import() and await in conjunction with import.meta.require to allow such evaluation ordering, but that seems a problematic integration/migration story.

---

This all means, if we can iron out that even with this PR that importing CJS is needed to fulfill various use cases, we should discuss the value that this PR brings. In particular, with the ability to import CJS, you can obtain a require() function by importing it from CJS:

import cjs from './cjs.js';
const {require, __filename, __dirname} = cjs.require;

module.exports = {require, __filename, __dirname};

This PR reduces the number of modules in the graph and reduces the boilerplate above. It also encourages people to use CJS by intentionally exposing the require function. We should focus on the reduction in boilerplate that this provides, and the intention of keeping require as a vital part of ESM modules in Node.

---

On the note of @mAAdhaTTah . I don't fully understand the comment. Given the ability to import CJS why would you not want to import React if it had both ESM and CJS? Why would you not want to import a library if it was only CJS? If path searching is preserved for example, you can even have a migration path that does not require that people select which format the library is in. import 'react'; is able to refer to both ESM and CJS. If the path searching algorithm selects ESM prior to CJS / JSON / etc. you would still get the ESM build.

---

On the note of @guybedford , I agree that this PR would be problematic if we were to remove the ability to import CJS and that import.meta.require is harder to analyze than import. Given my statements above, I do not think we can remove the ability to import CJS with this PR as it does not fulfill all the uses of being able to import CJS. I do think we could make it have fewer bailout conditions if we make import.meta.require non-configurable, but agree that it will still be very hard to analyze.

---

Finally, regarding minimalism; I think we should be very concise in what we want to ship as a minimal implementation. I think that this PR has merits, but given that I see it as having fewer uses vs the ability to import CJS and mostly appearing to be a boilerplate removal I would prefer it be discussed as a follow on to any initial implementation rather part of the initial implementation.

[mAAdhaTTah]

@bmeck My apologies for the confusion; the excerpt I quoted could have used more context. I was specifically addressing the browser interop concern raised by @guybedford.

I'm suggesting that import.meta.require doesn't impact browser interop because regardless of whether you import or import.meta.require a CJS module, a build step will be required to make it work in the browser. You're not going to run into a situation where you'd map the browser import statement to an ESM module and the Node import statement to a CJS one. If you have an ESM module, you'd just use it both.

[weswigham]

If the goal of import.meta.require is to provide a way to bail on using esm semantics and fallback to commonjs behaviors (so you can do less work to port older code), removing all the members on require seems like a mistake - people depend on require's ancillary functions; that's why they're there (and I'm going to reiterate this over and over again, exposing things to esm is not a "good time" to drop older APIs - either its for compatibility or its not). If import.meta.require isn't actually require, then it's not nearly as useful for porting code - it's a 3rd kind of code-running construct, separate from require in cjs or imports in esm. If all it can do is run cjs code, its only use is as a way to... run cjs - meaning if implemented without its members, the only purpose is to synchronously run cjs or attempt to subsume import's job of running cjs.

So it's either a bad polyfill because it's missing elements of the polyfill or a redundant entrypoint as imports can do the same thing. So at least one of those should be true for it to have value; either import shouldn't work on cjs (👎 ) , or it should have all the members a cjs consumer would expect require to have to it presents the same API it used to (despite being at a different identifier in esm for unfathomable reasons).

[giltayar]

Transparent interop (i.e. the ability to import a CJS module) enables a developer to move all their requires to import-s easily (and one can argue whether this is a good or a bad thing).

This is true, except for one case—when the require is inside a function and not a "top level" require. In this case, changing the synchronous require to an async await import (or the equivalent promise code) will turn that function from a synchronous function to an async one, which may mean a significant refactoring of the code base, and which will impede the migration of that codebase to ESM.

import.meta.require will help this migration by enabling the migration process to procede in an incremental fashion and not require major refactorings in the initial—and mostly technical—phase, of turning all require-s into import-s.

So, if it's not already obvious, I'm a big +1 on this.

[bmeck]

@giltayar this can be alleviated as I describe above by importing a require() function from a CJS file. I agree that there is merit to this, but I think we can leave this out of the initial implementation and see how common it is for people to use the functionality of require() vs import. Even if the require() is in a nested function scope, it may be fine to use import:

function foo(cb) {
  require('bar')(cb);
}

function foo(cb) {
  import('bar').then((ns) => ns.default(cb));
}

In particular I want to find how much of the usage is lazy loading regardless of timing, versus lazy loading that must be done synchronously. Even still, sometimes lazy loading is done due to writing things inline, not a requirement on the lazy loading itself.

In case it is not clear, I am -1 since this use case can be fulfilled with the ability to import CJS still. The boilerplate is heavier to import a require() function, but the ability to lazy load is still there through import() and the main question is if people need the lazy loading to be synchronous in a large enough usage.

[Addendum:] I would like to note that require() would require all lazy loaded dependencies be CJS in order to load synchronously.