ESM support: soliciting feedback

[cspotcode]

Please use this ticket to provide feedback on our native ESM support. Your involvement is greatly appreciated to ensure the feature works on real-world projects.

Experimental warning

Node's loader hooks are EXPERIMENTAL and subject to change. ts-node's ESM support is as stable as it can be, but it relies on APIs which node can and will break in new versions of node.

When node breaks their APIs, it breaks loaders using their APIs. You have been warned!

Third-party docs: "Guide: ES Modules in NodeJS"

Someone has been maintaining a great reference document explaining how to use ts-node's ESM loader.

Guide: ES Modules in NodeJS

First-party docs

Our website explains the basics:

CommonJS vs native ECMAScript modules
Options: esm

Usage

Requirements

• Set "module": "ESNext" or "ES2015" so that TypeScript emits import/export syntax.
• Set "type": "module" in your package.json, which is required to tell node that .js files are ESM instead of CommonJS. To be compatible with editors, the compiler, and the TypeScript ecosystem, we cannot name our source files .mts nor .mjs.
• Include file extensions in your import statements, or pass --experimental-specifier-resolution=node Idiomatic TypeScript should import foo.ts as import 'foo.js'; TypeScript understands this.
  • The language service accepts configuration to include the file extension in automatically-written imports. In VSCode:
    

Invocation

ts-node-esm ./my-script.ts

ts-node --esm ./my-script.ts

# If you add "esm": true to your tsconfig, you can omit the CLI flag
ts-node ./my-script.ts

# If you must invoke node directly, pass --loader
node --loader ts-node/esm ./my-script.ts

# To force the use of a specific tsconfig.json, use the TS_NODE_PROJECT environment variable
TS_NODE_PROJECT="path/to/tsconfig.json" node --loader ts-node/esm ./my-script.ts

# To install the loader into a node-based CLI tool, use NODE_OPTIONS
NODE_OPTIONS='--loader ts-node/esm' greeter --config ./greeter.config.ts sayhello

ts-node-esm / --esm / "esm": true work by spawning a subprocess and passing it the --loader flag.

Configuration

When running ts-node --esm, ts-node-esm, or ts-node all CLI flags and configuration are parsed as normal. However, when passing --loader ts-node/esm, the following limitations apply:

• tsconfig.json is parsed.
• CLI flags are not parsed.
• Environment variables are parsed.
• ts-node must be installed locally, not globally. npm install ts-node or yarn add ts-node.

tsconfig will be resolved relative to process.cwd() or to TS_NODE_PROJECT. Specify ts-node options in your tsconfig file. For details, see our docs.

Use TS_NODE_PROJECT to tell ts-node to use a specific tsconfig, and put all ts-node options into this config file.

Versioning

As long as node's APIs are experimental, all changes to ESM support in ts-node, including breaking changes, will be released as minor or patch versions, NOT major versions. This conforms to semantic versioning's philosophy for version numbers lower than 1.0. Stable features will continue to be versioned as normal.

node's API change: v16.12.0, v17.0.0

Node made a breaking change in their ESM API in version 17, backported to 16.12.0. It may also be backported to 14 and 12.
This is the change: nodejs/node#37468

ts-node automatically supports both APIs, thanks to #1457. This relies on hard-coded version number checks. If/when this is backported to node 14 and 12, we will publish a new version of ts-node with the appropriate version number checks. Be sure you are always using the latest version of ts-node to avoid problems.

---

---

---

---

Note: things below this line may be out-of-date or inaccurate. These notes were used during initial implementation, but have not been updated since

Pending development work

• Make resolution lookup use our fs caches
• Create esm-script.mjs to do --script-mode?
  • Can read process.argv for config resolution?
• Implement require('ts-node').esmImport(module, 'import-path')
  • use https://npm.im/tsimportlib
• Throw error when CJS attempts to require ESM, matching node's behavior for .js
  • See below: "Changes to existing functionality" > "require() hook"

The proposal

Below is the official proposal, explaining our implementation in detail.

---

I am asking node's modules team questions here: nodejs/modules#351

I was reading the threads about ESM support in ts-node, e.g. #935.

The @K-FOSS/TS-ESNode implementation is unfortunately incomplete; it does not attempt to typecheck. (it uses transpileModule)

So I did some research. Below is a proposal for ESM support in ts-node, describing the required behavior in detail.

This doesn't feel like an urgent feature to me, but I like having an official proposal we can work on.

---

Usage

node --loader ts-node/esm ./entrypoint.ts

Cannot be invoked as ts-node because it requires node flags; hooks cannot be enabled at runtime. This is unavoidable.

For simplicity, --require ts-node/register can be eliminated, because ts-node/esm automatically does that.

Alternatively, we publish an experimental ts-node-esm entry-point which invokes a node subprocess.

---

Don't forget allowJs! Affects the treatment of .js files. (Not .mjs nor .cjs because the TS language service won't look at them)

ESM hooks

Must implement ESM hooks to resolve extensionless imports to .ts files, resolve .js to .ts, classify .ts(x) and .jsx files as CJS or MJS, and compile .ts(x) and .jsx files.

resolve() hook:

Match additional file extensions: .ts, .tsx, .jsx.

Resolve .ts, .tsx, and .jsx if the import specifier says .js. Obey preferTsExts when doing this.

_

[Good idea?] Always ask default resolver first. If it finds something, we should not interfere.

--experimental-specifier-resolution=node does not obey require.extensions, unfortunately, so we can't use that.

getFormat hook:

If the resolved file is .ts, .tsx, or .jsx, behave as if extension was .js: use node's package.json discovery behavior to figure out if ESM or CJS.

This can be accomplished by appending .js to the URL path and delegating to built-in getFormat hook.

transformSource hook:

Same as today's code transformer. Relies on projects to be configured correctly for import/export emit.

Changes to existing functionality

require() hook

• Use same getFormat logic to determine if node will treat file as CJS or ESM.
• NOTE node already detects and throws some errors on its own. But if require.resolve points to a .ts file, we need to make the determination.
• If ESM, throw the same error as NodeJS ("cannot load ESM via require()")
  • copying logic from node's source here

require() code transform

• Must somehow allow import() calls.
• Force consumers to use require('ts-node').esmImport(module, 'import-path')?

ts-node bin entry-point

ts-node CLI does NOT need to support import()ing ESM.

WHY? Because ESM hooks are an experimental feature which must be enabled via node CLI flag.

Thus we will be loaded via --require, and Node is responsible for loading the entry-point, either triggering our hook or our require.extensions.

Allow import() in CJS

If "module": "commonjs", compiler transforms import() into __importStar

No way to change this without a custom transformer, which IMO is too much complexity at this time.

Users should run their code as ESM.

If they can't do that, we can recommend the following workaround:

// This is in a CommonJS file:
const dynamicallyImportedEsmModule = await require('ts-node').importESM('./specifier-of-esm-module', module);

Emit considerations

NOTE we have not implemented the following, although initially I thought we might. Instead, we assume tsconfig is configured for either ESM or CJS as needed

We could intelligently emit both "module": "esnext" and "module": "commonjs" depending on the classification of a file.

In transpile-only mode this is simple. Call transpileModule with different options.

When typechecking, we can pull SourceFile ASTs from the language service / incremental compiler.

We'll need a second compiler, one for each emit format. Or we can hack it by using transpileModule for all ESM output. transpileModule is incompatible with certain kinds of TS code, (can't do const enums) but it might work for a first-pass implementation.

[cspotcode]

TODO: turns out, users can tell the language service to include the .js file extension with automatically-written imports. So we do not need to automatically add them, though we do need to check if a .js import might point to a .ts or .tsx file.

The option is passed to the language service in a ts.UserPreferences object.
https://discordapp.com/channels/508357248330760243/640177429775777792/703301413337432114

[cspotcode]

I was trying to figure out if ts-node needs to automatically switch the "module" option between CommonJS and ESNext depending if we need to emit CommonJS or ESM. I concluded we do not want to do this. Here's an explanation anyway, in case I am proven wrong.

Today, ts-node respects the tsconfig's module option. Users are required to set it appropriately. If the user incorrectly sets module to ESNext and then tries to require() a TS file, they get an error because the emitted code has import statements.

Alternatively, we can automatically override the module option to be CommonJS when emitting for require() and ESNext when emitting for ESM. This allows a single tsconfig to be used for both ESM and CommonJS.

After thinking about this, it doesn't make sense. Users will choose either ESM or CommonJS via their package.json file. They won't do a mix of both. Also, this would get pretty messy since we'd be doing something that doesn't match tsc's output.

Nevertheless, if we wanted to implement this:

If the module option is already correct, we can use the languageService's getEmitOutput() like we do today. If not, we can grab a reference to the SourceFile and transform it using the same technique as transpileModule's implementation. This allow custom emit while avoiding an expensive parse.

TypeScript has an internal sourceFileAffectingCompilerOptions array. If any of those options differ, a SourceFile cannot be reused. However, some are only relevant if you care about diagnostics. For swapping out the module flag, I think SourceFile can always be reused.

[cspotcode]

We have released an experimental implementation of this in v8.10.1. Please test and share your feedback here.

[chpeters]

Thanks @cspotcode for the release! Everything seems be working minus one snafu. Importing named exports don't seem to be working, but this may be a Node module quirk. For example in index.ts:

import { graphql } from 'graphql';

will cause a syntax error of:

SyntaxError: The requested module 'graphql' does not provide an export named 'graphql'

but this can be solved by using destructuring:

import Graphql from 'graphql';
const { graphql } = Graphql;

Any way to support importing named exports in ts files?