Request: Allow specifying lookup paths for require.resolve

[nzakas]

tl;dr I'd like to propose an additional API be exposed to Node.js JavaScript that allows developers to perform require.resolve(), but with an custom list of lookup paths.

Background

In ESLint, we allow people to extend their configuration files with Node.js packages, so you can do something like this:

extends: airbnb

And then ESLint requires eslint-config-airbnb. That worked fine until people started creating shareable configs, published on npm, that wanted to inherit from other shareable configs (also published on npm). In that case, they'd want the require lookup to happen from their config file and not from the ESLint file doing the require.

Our Solution

What we ended up doing to get this to work: https://github.com/eslint/eslint/blob/master/lib/util/module-resolver.js

In short:

1. Create the normal array of lookup paths by combining Module.globalPaths and module.paths
2. Prepend another path to the front of that array
3. Use Module._findPath() to do the lookup, passing the lookup paths array

The obvious ugliness here is that we're not just relying on Module, but we're relying on Module._findPath(), which seems to indicate that this method should not be used (assuming the underscore is intended to mean "private".

What I'd Like

Some way to get the functionality of Module._findPath() that is an official Node.js API that we can rely on without fear of it changing or being removed. Some possible options (definitely not exhaustive):

1. Allow a second argument to require.resolve() that allows you to pass an array of lookup paths.
2. Bless Module._findPath() by creating something like Module.resolveFromPaths() that calls Module._findPath() under the covers

Of course, these are just a few ideas I had in my head. Anything that accomplishes the same functionality would be awesome.

Prior Art

It seems like there's a larger need for this capability based on modules available on npm:

1. resolve - effectively does the same thing by trying to recreate the Node.js lookup process. We tried using this, but there are enough differences with the real Node.js implementation that we abandoned it. It also hardcodes some information that should be dynamic (like core module names). 7 million downloads in the past month.
2. resolve-module - a simpler and less popular version of the same thing.
3. custom-resolve - a customized version of resolve.
4. resolve-cwd - require.resolve using CWD as the root. 50,000 downloads in the past month.
5. resolve-from - require.resolve from any arbitrary path. 1.5 million downloads in the past month.

[benjamingr]

Sorry, Module is locked. I don't think this can happen. I think it's worth discussing regardless.

---

I'm not sure I understand your particular problem, is this not something module.require (and not require) would solve for you?

If you don't want to require the actual file - is this something a module.require.resolve would fix for you?

[nzakas]

require.resolve could handle it if it could be given different lookup paths. We use the full path to various references that get passed around multiple files, so where we resolve the module path isn't the same place that we require it.

If Module is locked, does that mean it's safe to rely on Module._findPaths? If so, then maybe no change is needed.

[benjamingr]

_findPaths is an internal method, there is no gurantee regarding it as far as I know. In practice I doubt it'll break but that's not a position I'd be comfortable to have you in (relying on it).

Would you mind elaborating a little on why you need to pass the paths inside rather than using module.require ?

[nzakas]

Sure, basically, we have a configuration structure that we build up from multiple sources. At different points during that process, we convert Node.js module names into their paths, and we require those paths at different points in time. For instance, with extends, we do end up requiring fairly soon after resolving, but for parser we do not. The value of parser is passed along in the configuration (which contains only json-compatible data) until the very last moment before the JS code is parsed. At that point, it is both required to start parsing and the path is also passed along to rules in case they want to require it.

So, changing just require helps with one case but not the other.

[cjihrig]

FWIW, I'd also like to bless Module._findPath(), and will gladly submit the PR to make it happen.

[Trott]

Putting aside the Locked status issue for now, I'd also be for it (what @cjihrig said) if we can rename it to something without a _ and keep _findPath() as an undocumented alias for backwards compatibility.

Returning to the Locked status, if we have to vote to move it to Stable in the CTC or something, that's a high bar but not an entirely insurmountable obstacle. I certainly appreciate the effort here to document the use case because that should certainly be a pre-requisite for unlocking.

[benjamingr]

I think we can expose this functionality too (sans the _) as it directly relates to the internal module system and it's useful and there is no harm generally in doing it.

That said, we should also investigate what exposing it publicly would mean. What happens if someone overrides it with their own hooks? Should that change the internal reference or only further direct calls to it from outside of core?

Unlocking a module should require a very high bar.

Also - do we have any info on who else is using this from the outside?

[vkurchatkin]

It's not clear, why is this a problem at all. Algorithm is fixed, so implementing it in userland should be possible

[cjihrig]

It's fixed in theory. Things do actually change in it from time to time. I think @nzakas has made a pretty compelling argument that people want this behavior. It's already exposed, so I don't think committing to it is a bad idea.

[Trott]

It's already exposed, so I don't think committing to it is a bad idea.

That's the crux for me. It's already exposed, people are already using it. Exposing "private" methods and expecting people not to use them is something that we should try to correct where we can, and it seems that we can here.

[vkurchatkin]

@Trott the whole module is "private"

is something that we should try to correct where we can, and it seems that we can here.

We can't just document everything that's exposed.

[Trott]

@Trott the whole module is "private"

True, it's "private". It's also:

• exposed
• being used by people
• something the project is saying will not change
• the subject of a valid use case (at least for _findPath())

Those things taken as a whole seem to suggest that documenting _findPath() is both a low-risk and user-friendly approach.

There are other considerations, of course, and this may not be the right path after all. I don't see that as a foregone conclusion.

[vkurchatkin]

exposed

A lot of things are exposed.

being used by people

People use everything they can find

something the project is saying will not change

Public APIs are not going to change, and behaviour is not going to change. Stuff like _findPath doesn't even have well defined semantics. If we just take a random internal function, make it public and commit to never change it, we might found ourselves in trouble later.