`npx` invocation yields `ERR_INVALID_MODULE_SPECIFIER `

[brianjenkins94]

(@cspotcode This is from a while back on Discord)

Search Terms

• npx

Context

I run my typescript (served via npx) through ts-node so I don't have to maintain a build.

For an example of how I had this working pre-ESM, see this gist.

Expected Behavior

The program runs without error. In the case of the linked gist, it runs hello.ts, printing "Hello, world!".

Actual Behavior

$ npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d
Need to install the following packages:
  gist:d73578d7439427a59df28c1bc751e02d
Ok to proceed? (y) y

> node --experimental-specifier-resolution=node --loader=/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/ts-node/esm /Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts

(node:35543) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
node:internal/errors:464
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/ts-node/esm' imported from /Users/bjenkins/Desktop/foo/
    at new NodeError (node:internal/errors:371:5)
    at finalizeResolution (node:internal/modules/esm/resolve:405:11)
    at moduleResolve (node:internal/modules/esm/resolve:932:10)
    at defaultResolve (node:internal/modules/esm/resolve:1044:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:422:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:222:40)
    at ESMLoader.import (node:internal/modules/esm/loader:276:22)
    at initializeLoader (node:internal/process/esm_loader:74:49)
    at loadESM (node:internal/process/esm_loader:87:11)
    at runMainESM (node:internal/modules/run_main:47:21) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Steps to reproduce the problem

npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

Minimal reproduction

https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

Specifications

• ts-node version: v10.4.0
• node version: v16.13.2
• TypeScript version: v4.5.4

tsconfig.json

{
	"compilerOptions": {
		"allowJs": true,
		"esModuleInterop": true,
		"forceConsistentCasingInFileNames": false,
		"incremental": true,
		"isolatedModules": true,
		"jsx": "preserve",
		"lib": [
			"dom",
			"dom.iterable",
			"esnext"
		],
		"module": "ESNext",
		"moduleResolution": "Node",
		"noEmit": true,
		"resolveJsonModule": true,
		"skipLibCheck": true,
		"strict": false,
		"alwaysStrict": true,
		//"exactOptionalPropertyTypes": true,
		//"strictNullChecks": true,
		"strictBindCallApply": true,
		"strictFunctionTypes": true,
		//"strictPropertyInitialization": true,
		//"noImplicitAny": true,
		//"noImplicitThis": true,
		//"useUnknownInCatchVariables": true,
		"noImplicitOverride": true,
		"noImplicitReturns": true,
		"noPropertyAccessFromIndexSignature": true,
		"target": "ES2020"
	},
	"include": [
		"next-env.d.ts",
		"**/*.ts",
		"**/*.tsx"
	],
	"exclude": [
		"node_modules",
		"scripts/prev.ts"
	]
}

• Operating system and version: macOS Monterey 12.1 (21C52)

[cspotcode]

I think this path might be wrong:

https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d#file-npx-js-L49

When I look at the log output you shared, these are some of the paths I see:

/Users/bjenkins/Desktop/foo/
/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts
/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/ts-node/esm

The first, Desktop/foo, appears to be your working directory.
The second shows that npx is installing modules into ./caa40e90fbe6bfad/node_modules
So I assume that ts-node is also being installed into ./caa40e90fbe6bfad/node_modules.

This would explain why it cannot find caa40e90fbe6bfad/ts-node/esm, because ts-node is not installed there.

[brianjenkins94]

Whoops, fixed the path and now I get: ERR_INVALID_MODULE_SPECIFIER.

Note: You'll need to clear your npx cache: rm -rf ~/.npm/_npx/*

[cspotcode]

Whoops, fixed the path and now I get: ERR_INVALID_MODULE_SPECIFIER.

Are you able to share the full error, including npx invocation and stack trace?

[brianjenkins94]

$ npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

> node --experimental-specifier-resolution=node --loader=/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/ts-node/esm /Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts

(node:38413) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:38413) ExperimentalWarning: The Node.js specifier resolution in ESM is experimental.
(node:38413) ExperimentalWarning: The Node.js specifier resolution in ESM is experimental.
TypeError [ERR_INVALID_MODULE_SPECIFIER]: Invalid module "file:///Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts" 
    at new NodeError (node:internal/errors:371:5)
    at ESMLoader.load (node:internal/modules/esm/loader:324:13)
    at async ESMLoader.moduleProvider (node:internal/modules/esm/loader:230:47)
    at async link (node:internal/modules/esm/module_job:67:21) {
  code: 'ERR_INVALID_MODULE_SPECIFIER'
}

[cspotcode]

Thanks, I can take a closer look tonight. In the interim, I see that you're using ts-node 10.4.0. It may be worth confirming that 10.5.0 exhibits the same behavior.

[brianjenkins94]

Same result by the looks of it:

$ npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d

> node --experimental-specifier-resolution=node --loader=/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/ts-node/esm /Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts

(node:38772) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:38772) ExperimentalWarning: The Node.js specifier resolution in ESM is experimental.
(node:38772) ExperimentalWarning: The Node.js specifier resolution in ESM is experimental.
TypeError [ERR_INVALID_MODULE_SPECIFIER]: Invalid module "file:///Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts" 
    at new NodeError (node:internal/errors:371:5)
    at ESMLoader.load (node:internal/modules/esm/loader:324:13)
    at async ESMLoader.moduleProvider (node:internal/modules/esm/loader:230:47)
    at async link (node:internal/modules/esm/module_job:67:21) {
  code: 'ERR_INVALID_MODULE_SPECIFIER'
}

[cspotcode]

I am seeing that the path may still be wrong.

Instead of

/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/ts-node/esm

it should be

/Users/bjenkins/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/node_modules/ts-node/esm

[cspotcode]

Also, you might need to include the .mjs extension in ts-node/esm.mjs since you're specifying an absolute path. I'm not sure

[cspotcode]

@brianjenkins94

I figured it out. The problem is primarily:

a) hello.ts file is ignored

When you run stuff via npx, it gets installed into a node_modules directory. This is ignored by ts-node by default. specifying skipIgnore fixes this.

Since the file is ignored, it does not get special treatment by our ESM loader. Node's default behaviour is used, and node does not understand the .ts file extension.

Explained here:
https://typestrong.org/ts-node/docs/how-it-works/#skipping-node_modules

---

If you configure skipIgnore via your tsconfig.json, you may also hit a second problem:

b) we can't automatically pick up your tsconfig.json

This is a limitation of invoking node --loader that will be avoided with #1655

When you run ts-node hello.ts, you're invoking our bin entrypoint, so we are able to locate hello.ts and find your tsconfig.json relative to it, before we execute the entrypoint. However, when you do node --loader ts-node/esm.mjs we have to load tsconfig based on your cwd before we receive the path to hello.ts. When you're using npx your cwd is very different from where npx is installing stuff. We may find a different tsconfig in your cwd, or we may not find any tsconfig and fallback to default settings.

Your npx.js bootstrapper can avoid this by setting the TS_NODE_PROJECT environment variable.

[brianjenkins94]

Is it possible to pass options to the loader in the command? Like so?:

node --experimental-specifier-resolution=node --loader=ts-node/esm --skipIgnore hello.ts
                                                                   ^^^^^^^^^^^^

I couldn't really tell from the docs. Ideally I could set the --project flag the same way.

[cspotcode]

It's not, because you're invoking node, not ts-node, so those flags all go to node, and it'll throw an error that it does not understand them.

…

On Tue, Mar 1, 2022, 2:53 PM Brian Jenkins ***@***.***> wrote: Is it possible to pass options to the loader in the command? Like so?: node --experimental-specifier-resolution=node --loader=ts-node/esm --skipIgnore hello.ts ^^^^^^^^^^^^ I couldn't really tell from the docs. Ideally I could set the --project flag the same way. — Reply to this email directly, view it on GitHub <#1663 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC35ODRVT5XZ45V4DEEMNDU5ZYTFANCNFSM5PPQV52Q> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[brianjenkins94]

Huzzah, it works!

$ npx https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d 

> node --experimental-specifier-resolution=node --loader=/Users/bjenks/.npm/_npx/caa40e90fbe6bfad/node_modules/ts-node/esm /Users/bjenks/.npm/_npx/caa40e90fbe6bfad/node_modules/sampleThatReproducesTheIssue/hello.ts

Success!

It doesn't print out "Hello, world!" for some reason but I can dig into that on my own.

Thanks so much for your help, @cspotcode !

For the future reader, here's what worked:

https://gist.github.com/brianjenkins94/d73578d7439427a59df28c1bc751e02d#file-npx-js-L10-L53