Clarity on how to determine progmatically whether a sourceFile is to be output as "commonjs" or "esm".

[craigphicks]

Bug Report

It appears that at least some import path "before"-transform action that works with "commonjs" mode, needs "after"-transform action to work in "esm" mode. That's OK, and not a bug.

For example

import {foo} from '@xxx/foobar`
import type {Foo} from '@xxx/foobar`

where the before-transform replaces '@xxx/foobar with './foobar`.
That's fine in "commonjs" and the final results looks like

const foo_1 = require("./foobar");

In "esm", still using the before-transform, I try replacing '@xxx/foobar with './foobar.js`,
but the result looks like

import {foo} from './foobar.js'
import type {Foo} from 'foobar.js'

and the import type is an error.
Fortunately, just changing the before-transform to an after-transform for the "esm" case only, works perfectly.
So that is not a bug.

Now the problem is only determining whether typescript is going to output "commonjs" or "esm" (or something else).

Currently I am using program.getCompilerOptions().module>=ts.ModuleKind.ES2015 to decide if a ".ts" file will be output as "esm"
but I am not absolutely confident that is correct. This is based on the documentation for compilerOptions/module and the assumption that going forward all new values for module will imply "esm".

Now looking at the experimental doc for esm, it appears that
if a file extension is .cts or .mts then it will be "commonjs" or "esm" respectively, whatever the module value. Although they won't appear as source files unless module is set to 'node12' or 'nodenext', I guess.

But what's the difference between 'node12' and 'nodenext'? Is it the default output type of straight .ts extension file (commonjs vs esm)? The document doesn't say. The document talks much about package.json/type - that is not being used as the source of truth for the default output (commonjs vs esm), is it?.

This report is classifying the lack of documentation clarity as a bug,
because without clarity there is a good change of written software crashing.

🔎 Search Terms

import path transform commonjs vs esm modules

🕗 Version & Regression Information

I noticed when reading documentation for compilerOptions/module, experimental doc for esm.

⏯ Playground Link

💻 Code

🙁 Actual behavior

🙂 Expected behavior

[andrewbranch]

The ESM doc says

Node.js supports a new setting in package.json called type. "type" can be set to either "module" or "commonjs".... This setting controls whether .js files are interpreted as ES modules or CommonJS modules, and defaults to CommonJS when not set.... To overlay the way TypeScript works in this system, .ts and .tsx files now work the same way. When TypeScript finds a .ts, .tsx, .js, or .jsx file, it will walk up looking for a package.json to see whether that file is an ES module...

The type field in package.json is nice because it allows us to continue using the .ts and .js file extensions which can be convenient; however, you will occasionally need to write a file that differs from what type specifies. You might also just prefer to always be explicit.

Node.js supports two extensions to help with this: .mjs and .cjs. .mjs files are always ES modules, and .cjs files are always CommonJS modules, and there’s no way to override these.

In turn, TypeScript supports two new source file extensions: .mts and .cts. When TypeScript emits these to JavaScript files, it will emit them to .mjs and .cjs respectively.

Doesn’t that spell everything out? And yes, at the top you’re supposed to understand that the document is in context of node12 and nodenext. The difference between the two is simply which resolution features are enabled based on what different versions of Node support, same as targeting es2018 vs. es2020.

[craigphicks]

@andrewbranch - Thank you very much for taking the time to answer.

Let me phrase the problem in the operational terms. During a solution watch build, a piece of software for emit-time custom transformations needs to know, on a per-emit, per-SourceFile basis, whether typescript is going to transform the SourceFile as commonjs or esm. (In the use-case described, it will process as a "before"-transform or "after"-transform accordingly.)

If SourceFile["fileName"] has a .cts or .mts extensions it is simple to decide.

Does your answer mean that in the case of .ts extensions, the package.json/type property should be be treated as the ultimate source of (commonjs/module) truth? (I think "yes".)

The reason I feel some dissonance in that answer is because the package.json information, as far as I know, is not passed in runtime per-emit parameters - CompilerOptions are available via Program["getCompilerOptions"] but there is no counterpart for package.json, (or am I wrong)?

Also, the SolutionBuilderWithWatchHost framework (which I am using) doesn't seem to include a projects package.json as a file watch-dependecy, but of course the tsconfig file is a watched dependency (confirmed in v4.4.4, v4.5.5). I had assumed that to mean that package.json was not an ultimate source of truth for anything. (Note: the tsc program differs from SolutionBuilderWithWatchHost with respect to some watch-dependencies, I'm not sure at this moment whether tsc watches package.json or not). But of course my assumption could be wrong - it could be that SolutionBuilderWithWatchHost uses package.json as source of (commonjs/module) truth, reads it only once at startup, and assumes it will not change.

What should a client application of the SolutionBuilderWithWatchHost API do to ensure it correctly mirrors the (commonjs/module) truth that SolutionBuilderWithWatchHost itself is using? (And what about the more general not-only-SolutionBuilderWithWatchHost case?).

[andrewbranch]

Ah, you’re asking for an API to answer the question more than you’re asking how it works. ts.getImpliedNodeFormatForFile is what you’re after. It will indeed search for package.json files on disk, so you should try to pass it the moduleResolutionCache.getPackageJsonInfoCache() that was used in program construction to avoid those FS hits if performance is important. I believe any package.json actually queried in this process gets watched, but it’s possible something was missed in the SolutionBuilderWithWatchHost APIs. @weswigham or @sheetalkamat might know more about that.

[weswigham]

During a solution watch build, a piece of software for emit-time custom transformations needs to know, on a per-emit, per-SourceFile basis, whether typescript is going to transform the SourceFile as commonjs or esm. (In the use-case described, it will process as a "before"-transform or "after"-transform accordingly.)

We set SourceFile.impliedNodeFormat with this value once we calculate it during program construction, but the field is internal. Its calculation is fairly involved, since you gotta look at the package.json context for the file, as @andrewbranch says.

[craigphicks]

@andrewbranch @weswigham - Wonderful - thank you so much!

In ts 4.5.5 I can see the member SourceFile["impliedNodeFormat"] so it is public. (ts-expose-internals is not necessary to see it). Hovering shows the documentation:

(property) SourceFile.impliedNodeFormat?: ts.ModuleKind.CommonJS | ts.ModuleKind.ESNext | undefined
When module is Node12 or NodeNext, this field controls whether the source file in question is an 
ESNext-output-format file, or a CommonJS-output-format module. This is derived by the module 
resolver as it looks up the file, since it is derived from either the file extension of the module, or 
the containing package.json context, and affects both checking and emit.

It is public so that (pre)transformers can set this field, since it switches the builtin node module 
transform. Generally speaking, if unset, the field is treated as though it is ModuleKind.CommonJS.

Only the last sentence "Generally speaking ..." is ambiguous, from the perspective of an API client application.

I think this is a correct future-proof coding for an API client -

• if SourceFile.impliedNodeFormat===ModuleKind.CommonJS, then "commonjs"
• else if SourceFile.impliedNodeFormat===ModuleKind.ESNext, then "module"
• else if compilerOptions.module===ts.ModuleKind.CommonJS, then "commonjs"
• else if compilerOptions.module>=ts.ModuleKind.ES2015, then "module"
• else it is neither "commonjs" nor "module"

Would you be so kind as to confirm whether it is correct? If it is correct, I will close this issue.

[andrewbranch]

No—the algorithm is

• if sourceFile.impliedNodeFormat === ModuleKind.ESNext, then "module"
• else "commonjs"

[andrewbranch]

I think the code comment could just as well have left off “Generally speaking”

[weswigham]

The module compiler option matters most - the impliedNodeFormat field is unset (and its transforms not supplied) if module is not node12 or nodenext, and it won't be read by emit if its accompanying transform isn't in the pipeline :)

[weswigham]

I think the code comment could just as well have left off “Generally speaking”

I mean, non-module (script) source files technically exist. They're just rare.

[andrewbranch]

Oh yeah sorry, my comment was assuming node12/nodenext but that’s not in your assumptions. As for scripts, you can tell modules from scripts by the source file’s externalModuleIndicator (and commonJsModuleIndicator in JS files), but those fields are internal. There’s ts.isExternalModule() if you’re only concerned with TS. Like Wes said, if your module is ES2015+ and your input code looks like an ES module, the output code will be an ES module. If your module is CommonJS and your input code looks like an ES module or CJS module, your output code will be a CJS module.

[andrewbranch]

I think this is a correct future-proof coding for an API client

In other words, I think this is right if you assume all input files are modules, and ignore some weird things like --module none which does sometimes incorrectly produce CommonJS modules 😅

[craigphicks]

@andrewbranch @weswigham - Great! This has been a huge help! Closing now.