ts-node fails when ES Modules are in the dependency graph in Node.js 13+

I basically detailed the issue in this comment: #155 (comment)

It's a chicken-and-egg-like problem:

If we use module: 'commonjs', then if any TS files import ES Modules (indirectly in their dependency graph), then Node throws an error because CommonJS modules can not import ES Modules.
If we change to module: 'esnext', then the errors from the previous point go away, but now the .js file that is loading ts-node and calling something like require('typescript-entry-point.ts') will have a similar issue, because the call to require('typescript-entry-point.ts') will try to load an ES Module.
Finally, if we convert the .js file into an ES Module, we can not convert require('typescript-entry-point.ts') into import 'typescript-entry-point.ts' because now ES Modules don't handle .ts extensions (at least not out of the box, and it seems the old require hooks don't operate on these new identifiers)

At the moment, I'm sort of stuck, because I have dependencies in my dependency tree that are ES Modules.

The only workaround I can think of is to compile everything to .js files (ES Modules) and avoid to use ts-node.

I wonder if a combination of allowJs and ignore so that it compiles JS files would help. I haven't tried that yet.

One way to reproduce this is to take a working application with module: 'commonjs' being passed to the register() call, make sure you're using the latest Node 13, then add an ES Module dependency (namely a .js file inside a node_modules package with type: 'module', as I believe that'll be the prime way that people will publish ES Modules from now on) on the leaf end of any branch of the dependency tree.

Now that Node 13 is out in the wild, looks like people are starting to run into this without any flags. Eventually someone will have unpaid free time to fix it, but I think it is easier to just pre-compile everything to JS before running Node.

If we use module: 'commonjs', then if any TS files import ES Modules (indirectly in their dependency graph), then Node throws an error because CommonJS modules can not import ES Modules.

I'm trying to understand this scenario - how did this work before? I assume existing TypeScript running via ts-node was never meant to use real ESM semantics, instead it's more similar to babel-style "using import as sugar for require". So it's de-facto CommonJS and shouldn't be loading ES modules from its dependencies (at least that should be the default behavior afaict).

Is this about existing projects breaking or about existing projects migrating from CJS-"import" to standard imports?

Ah, I think I got it now: It's about running ts-node for module code. Which would involve compilation within the ES module loader, potentially registering the compiler after node is started up (as opposed to using a CLI flag). The short answer is that the APIs to allow that aren't yet in node, so I don't think ts-node can do anything about this for now. :(

Not even by hooking in to the experimental load API?

@GeoffreyBooth

There's two independent holes in the API surface:

Being able to change the source text of something loaded from disk without breaking relative resolution. There is an open PR to address this by @GeoffreyBooth: esm: source hooks nodejs/node#30986
Being able to start a simple binary (ts-node) that registers a transformer and then runs user code. Right now the only way to register hooks is using a node flag (node --some-flag=ts-node main.ts). Short of forking a new node process, there's currently no API that allows to implement ts-node main.ts with native module support.

The second point is tricky because it's by design. We don't really want to allow userland code (from node's perspective) being able to switch out parts of the loading pipeline at runtime. It can lead to weird issues where the same global scope contains code expecting different module loaders and thanks to import() that's actually observable (as opposed to require).

One idea brought up by @guybedford was to have an API that can create a new global scope ("context" or "realm") with all bells and whistles, including node standard library, process global, and fresh module system. Tools like ts-node could use it to set up a new context for running the application code. But there are some downsides and questions, e.g. around garbage collection of the original "bootstrap" context and the time it would take to initialize the new context.

I created a proof of concept for using the current version of the --experimental-loader feature to transpile typescript files right before node loads it and then give node the now ESNext Javascript code with the SyntheticModule VM feature. It's fairly hacky right now but it made for a fun 6 hour session: https://github.com/KristianFJones/TS-ES-Node

^ Just for above there's a new loader hook in node nightly called getSource so the following loader can be used in node nightly. Only caveats are the same as the babel typescript plugin (e.g. no const enum):

Typescript Loader

import fs from 'fs';
import path from 'path';
import babel from '@babel/core';
import urlUtils from 'url';
import babelPluginTransformTypescript from '@babel/plugin-transform-typescript';

function isURL(string) {
    try {
        new URL(string);
        return true;
    } catch {
        return false;
    }
}
const removePrivateTypes = (babel) => {
    return {
        visitor: {
            ClassPrivateProperty(context) {
                context.get('typeAnnotation')?.remove();
            },
        },
    };
}

const BABEL_OPTS = {
    plugins: [
        babelPluginTransformTypescript,
        removePrivateTypes,
    ],
    parserOpts: {
        plugins: [
            'asyncGenerators',
            'bigInt',
            'classProperties',
            'classPrivateProperties',
            'importMeta',
            'nullishCoalescingOperator',
            'numericSeparator',
            'objectRestSpread',
            'optionalCatchBinding',
            'optionalChaining',
            'topLevelAwait',
            'typescript',
        ],
    },
    sourceMaps: 'inline',
};

export async function getSource(urlString, context, getSourceDefault) {
    if (isURL(urlString)) {
        const url = new URL(urlString);
        if (url.pathname.endsWith('.js') && !fs.existsSync(url)) {
            url.pathname = url.pathname.replace(/\.js$/u, '.ts');
            const contents = await fs.promises.readFile(url, 'utf8');
            const { code: source } = await babel.transformAsync(contents, {
                ...BABEL_OPTS,
                sourceFileName: path.basename(urlUtils.fileURLToPath(url)),
            })
            return { source };
        }
    }
    return getSourceDefault(urlString, context, getSourceDefault);
}

export async function resolve(specifier, context, defaultResolve) {
    try {
        const defaultResolution = defaultResolve(specifier, context, defaultResolve);

        try {
            const url = new URL(defaultResolution.url);
            url.pathname = url.pathname.replace(/\.ts$/, '.js');
            return { url: url.href };
        } catch {
            return defaultResolution;
        }
    } catch {
        return { url: new URL(specifier, context.parentURL).href };
    }
}

Hi, I created the getSource hook. There's also a transformSource hook that was intended for things like transpilation. See the transpiler loader example: https://github.com/nodejs/node/blob/master/doc/api/esm.md#transpiler-loader

Look's like I know what I'm doing this weekend.

Been busy, here is a proof of concept using the transformSource hook. Supports relative imports of .ts and .tsx without requiring extensions. https://github.com/K-FOSS/TS-ESNode

Okay, made a few small changes and fixes. It's now able to handle external modules that have the TSLib import helper without requiring refactoring existing imports. The TS-ESNode loader hook can be dropped into existing projects and just work. Hopefully! I just tested it on my main Application template that uses TypeORM & TypeGraphQL with tsconfig.json set to use ESNext as target and modules and it all just works. Transpiled code is all using imports and exports with Node V14 in Module mode.

Removed the globby requirement. Module has been published on NPM https://www.npmjs.com/package/@k-foss/ts-esnode zero dependencies other then peer dependency on TypeScript. NPM reports unpacked size of 8.66 kB. Should hopefully be a drop in replacement for TS-Node for now.

@KristianFJones If you're interested I'd love to land a PR in ts-node with this functionality.

mark.

@erick2014

I was trying to setup this typescript for apollo-server and I was getting the same error, this is how my tsconfig.json file end up:

{
  "compilerOptions": {
    "target": "es2020",
    "module": "esnext",
    "lib": ["es2020"],
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true,
    "moduleResolution": "node",
    "sourceMap": true
  },
  "ts-node": {
    "esm": true
  }
}

Thanks this actually worked @erick2014

Fix issue importing ESM modules

Please, can somebody explain for dummy how to use ts-node imports.
Whatever combination I'm trying to use resulting in some errors either
Cannot use import statement outside a module or any other import problem...

Trivial setup:
package.json

{
  "name": "items",
  "version": "0.1.0",
  "private": true,
  "type": "module",
  "scripts": {
    "test": "ts-node app/translate.ts"
  },
  "dependencies": {
  },
  "devDependencies": {
    "sass": "^1.63.2",
    "ts-node": "^10.9.1",
    "@types/node": "20.2.5",
    "globby": "^13.1.4",
    "typescript": "5.1.3"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "target": "es2020",
    "module": "esnext",
    "lib": ["es2020"],
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true,
    "moduleResolution": "node",
    "sourceMap": true
  },
  "include": ["app/*.ts"],
  "exclude": ["node_modules"],
  "ts-node": {
    "esm": true
  }
}

code app/translate.ts

import sass from "sass"
async function main() {
  const style = await sass.compileAsync('template/style.scss')
  console.log(style)
}
Promise.all([main()])

Error

> ts-node app/translate.ts

(node:27364) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
E:\Development\Items\node_modules\sass\sass.node.js:1
import * as util from "util"
^^^^^^

SyntaxError: Cannot use import statement outside a module

Will really appreciate for help.

@menangaTech

@menangaTech I was trying to use formadata-node, which in an ESM-only package. Here's a snippet that causes problems:
import { FormData } from "formdata-node"
const form = new FormData()
I fixed it by having "type":"module" in my package.json file and the following tsconfig.json:
{
  "compilerOptions": {
    "target": "es2016",
    "module": "nodenext",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  },
  "ts-node": {
    "esm": true
  }
}
The important parts are the module and ts-node settings

Answer after answer on StackOverflow said to remove "type":"module" from package.json, yet that setting (in addition to your tsconfig) solved all my issues. Thank you!

Mail.com / ipmartini@gmail.com / darknetencryption@mail.com

Any update on this?
ts-node will always fail when the "type": "module" is in package.json

Any update?
"type": "module" in package.json still failing

Any update?
"type": "module" in package.json still failing

You better stop using ts-node and start using tsx unless you really need to use the emit decorators metadata option of typescript if you're using something like TypeORM (you can still use tsx though).

I get it that the error is because some dependency became an ES Module when previously was CommonJS. I have a funny situation now that a project works, but if I remove package-lock.json and reinstall, it stops working. Clearly some sub-dependency has changed into ESM. Is there an easy way to detect which one it is so I can pinpoint it's version and maybe let the authors know?

@achlubek

@achlubek Almost always when I hit this message if you look very carefully at the error you receive it will allow you to track down which specific module is causing the problem.

@achlubek

@achlubek Almost always when I hit this message if you look very carefully at the error you receive it will allow you to track down which specific module is causing the problem.

Yeah I managed to track it down to Chai v5 becoming ESM only. Too bad eh!

Any update?
"type": "module" in package.json still failing

You better stop using ts-node and start using tsx unless you really need to use the emit decorators metadata option of typescript if you're using something like TypeORM (you can still use tsx though).

Any recommendation about using tsx with emitDecorators? Im having troubles with that.

Any update?
"type": "module" in package.json still failing

You better stop using ts-node and start using tsx unless you really need to use the emit decorators metadata option of typescript if you're using something like TypeORM (you can still use tsx though).

Any recommendation about using tsx with emitDecorators? Im having troubles with that.

tsx can't emit decorators, and I don't think it's possible (during development)

I fixed this by running npx tsx instead of npx ts-node

Once again, ES Modules are the dumbest thing to ever happen to the JS ecosystem

Navigation Menu

Repository navigation

`ts-node` fails when ES Modules are in the dependency graph in Node.js 13+ #935

61 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ts-node fails when ES Modules are in the dependency graph in Node.js 13+ #935

Description

Activity

trusktr commented on Jan 2, 2020

trusktr commented on Jan 2, 2020

hybrist commented on Jan 2, 2020

hybrist commented on Jan 2, 2020

trusktr commented on Jan 3, 2020

hybrist commented on Jan 3, 2020

EntraptaJ commented on Jan 4, 2020

Jamesernator commented on Jan 10, 2020

GeoffreyBooth commented on Jan 10, 2020

EntraptaJ commented on Jan 10, 2020

EntraptaJ commented on Jan 18, 2020

EntraptaJ commented on Jan 19, 2020

EntraptaJ commented on Jan 19, 2020

blakeembrey commented on Jan 20, 2020

61 remaining items

thomas-void0 commented on Apr 21, 2023

NeoPrint3D commented on Apr 29, 2023

eiva commented on Jun 11, 2023

kshep92 commented on Aug 11, 2023

Prenuer commented on Aug 13, 2023

samislam commented on Dec 5, 2023

Sengulair commented on Dec 29, 2023

samislam commented on Dec 29, 2023

adrian-afl commented on Dec 30, 2023

owenallenaz commented on Jan 3, 2024

adrian-afl commented on Jan 3, 2024

SamuelGaona commented on Feb 14, 2024

samislam commented on Feb 14, 2024

nabilfreeman commented on Jul 5, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`ts-node` fails when ES Modules are in the dependency graph in Node.js 13+ #935