Repository navigation
bug(esm): TypeScript is not an ECMAScript superset post-ES2015 #50501
Description
Activity
I stopped reading here:
The output program
T'will throw according to the ES spec.The ECMAScript specification says nothing about the meaning of module specifiers. Module resolution is the host’s responsibility and is therefore implementation-defined. If the TS compiler can resolve an import but your runtime can’t, that just means your project is incorrectly configured for the target runtime environment and doesn’t say anything about ES compliance.
If you want to argue this behavior is suboptimal, that’s fine, but you can’t use spec compliance as a bludgeon here.
Reacted by Joshua Chen, Martin Johns, Nicolò Ribaudo, Andrii Dieiev, Joost Koehoorn, Ryan Cavanaugh, Daniel Rosenwasser, mashaal, Christian Stewart, Robert Hurst and 9 moreReacted by Lewis, Sergio López Madriz, Homa Wong, Risto Keravuori, catmeme, Andrés Correa Casablanca, Michael Mykhaylov, Rob Resendez, Chris Ackerman, Michał Lytek and 4 moreSigh so many people thinking that explicit extensions and full paths are ES requirements is really Node's fault. They could well keep the CJS resolution mode and make everyone happy. (Same goes for JSON modules—soon people will think
assert { type: "json" }is required for JSON modules, just because Node is paranoid about aligning with browsers these days.)Also, if you use
ts-node, you could well make your code run with.jsextension imports via the resolver—it's host-defined behavior at the end of the day.Reacted by Daniel Rosenwasser, Georgii Dolzhykov and Matt KantorModule resolution is the host’s responsibility and is therefore implementation-defined
This is technically true (though in practice, pedantic).
The ECMAScript specification says nothing about the meaning of module specifiers.
This is not true:
Multiple different referencingScriptOrModule, specifier pairs may map to the same Module Record instance. The actual mapping semantic is host-defined but typically a normalization process is applied to specifier as part of the mapping process. A typical normalization process would include actions such as alphabetic case folding and expansion of relative and abbreviated path specifiers.
We can go round and round on this if you want, but whether you're in the browser, you're in Node, or any other major host of note, in pure ES,
import "./file.js"will be valid output andimport "./file"will not be.Reacted by Homa Wong, Andrés Correa Casablanca, AⱯ, Eugene Mirotin, Carlos G. T. Armenta Andrade, Iuliu Pop, Mike, Konstantin Shutkin, snarbies and Jason HydeReacted by Georgii Dolzhykov"Include actions such as" is extremely handwavy. It's neither normative nor actually specifying any algorithmic steps. It says nothing about
import "./file"being invalid orimport "./file.js"being valid.Reacted by Doug Moscrop, Georgii Dolzhykov, Paul Draper and DakotaReacted by snarbiesAn example of “typical” use does not constitute normative text.
Reacted by Doug Moscrop, Georgii Dolzhykov, Tyler Sebastian and DakotaReacted by snarbiesStockholm Syndrome ITT.
Reacted by mashaal, AⱯ, Carlos G. T. Armenta Andrade, Tyler Sebastian, Mike, Threebow and snarbiesReacted by Martin Johns, Christian Stewart, Ilia Pozdnyakov, Doug Moscrop, Chi Ma and DakotaReacted by Joshua Chen, Bruce Pascoe, Andrii Dieiev, Christian Stewart, Michael Mykhaylov, Carlos G. T. Armenta Andrade, Wilmar :) and ThreebowReacted by ThreebowWe've had thousands of people confused and upset over this for the last 5 years because their valid input programs end up failing to generate valid output.
On the basis of that alone you're wasting your time, you're not going to convince me that doing nothing is the appropriate solution, thanks for the comments though.
Reacted by Sergio López Madriz, mashaal, Andrés Correa Casablanca, Benedikt Strehle, Radoslav Kirilov, AⱯ, Eugene Mirotin, Carlos G. T. Armenta Andrade, Leon Adler, Cymaera and 15 moreOn the basis of that alone you're wasting your time, you're not going to convince me that doing nothing is the appropriate solution, thanks for the comments though.
Like I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here.
Reacted by Martin Johns, Daniel Rosenwasser, Christian Stewart, Dan Freeman, Georgii Dolzhykov and Chi MaReacted by snarbiesReacted by LewisLike I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here.
OK, fine, I take your point with the limited note that it doesn't say nothing about it, and also point to implementations in practice. There's enough dishonesty in this whole issue to go around. But yes I will grant you that the spec does not explicitly specify this - we'd probably agree that it could and should.
Though, to be clear,
import "./a"is not even target-legal for Node target. So I mostly dispute that there's a substantive issue with my statement.Reacted by Carlos G. T. Armenta AndradeReacted by Doug Moscrop- in my most recent vanilla browser module experiments, they are written in what language do you imagine? i’ll give you a tip it begins with j and ends with sOn Mon, 29 Aug 2022 at 04:42, C. Lewis ***@***.***> wrote: Like I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here. OK, fine, I take your point with the limited note that it doesn't say *nothing* about it, and also point to implementations in practice. There's enough dishonesty in this whole issue to go around. But yes I will grant you that the spec does not explicitly specify this - we'd probably agree that it could and should. — Reply to this email directly, view it on GitHub <#50501 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABU35SXLH6UQ4F65IEAC5LV3QPQRANCNFSM574CX7OQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>-- Kind regards *Nikos Katsikanis* <https://nikoskatsikanis.com> *Tech Innovation* | *Podcast Host | Linktr.ee <https://linktr.ee/nikosshow>*Reacted by Lewis, Christian Stewart, Tristan Hessell , catmeme, Ilia Pozdnyakov and Michael Mykhaylov
I'm probably going to regret wading into this, but I wouldn't frame this issue in terms of language supersets, if the goal is to change how
importstatements are emitted."TS is a superset of JS" means that every valid JS program is also a valid TS program, in a particular sense. It basically just means that a valid JS program should be emitted as-is (or possibly downleveled). It's quite a weak statement. The only violations of this I'm aware of are things like parsing edge cases as in #33639.
It doesn't imply that every valid TS program will be emitted as a valid JS program, although obviously such behavior is important. It might not be desirable to emit some line of TS as-is to JS, but doing so doesn't violate "TS is a superset of JS".
Reacted by Bruce Pascoe, Ryan Cavanaugh, Nicolò Ribaudo, Martin Johns, Nicholas Bailey, Christian Stewart, Daniel Rosenwasser, Mathias Schreck, Ilia Pozdnyakov, Leon Adler and 4 moreReacted by Lewis and AⱯLike I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here.
Let's forget about the specification then, given that those using TS dislike the current behaviour, and you are likely to accept it is suboptimal. How about we finally change it!
I really don't care if I have to use a config file and CLI argument at the same time to make it work.
It doesn't imply that every valid TS program will be emitted as a valid JS program, although obviously such behavior is important.
If you describe TS as a superset of JS, this is correct (in TS is JS with extra stuff, some of that could make it incompatible). However, the problem is that:
TypeScript adds optional types to JavaScript that support tools for large-scale JavaScript applications for any browser, for any host, on any OS. TypeScript compiles to readable, standards-based JavaScript.
If it should work in any browser, changing './foo' to './foo.js' is desirable. Unless you want to take the battle to the browsers, changing the current behaviour of TS looks like the way to go.
Reacted by Lewis, AⱯ, 风, staplespeter, Mike, Konstantin Shutkin and snarbiesThe whole idea of TypeScript is to add static types on top of JavaScript, not to be a higher-level language that builds to JS.
Not a fan of this backtracking either, 5 years ago this was not the rhetoric that brought me onto TS. TS offered much more, it provided design constructs that were lacking in js and also more expressive syntax (which eventually got added to js). But mostly, the design constructs of access modifiers, statics, interfaces and classes made it possible to design large systems without breaking a sweat. Maybe the differences in the ts implementation and eventual es standard is now deterring the team from adding new features to the language? Case in point,
#vsprivate. legacy decorators. different field initializers. So it looks like the goal post was (silently) moved? Maybe it is also to appeal to the masses of javascripters? I don't believe this is entirely good. TS used to show what JS could be. Now only babel can do that, but new syntax that compiles through babel is not compatible with the ts language server. So users have to chose between new features with babel on js vs ts with current features. Also babel is limited to whatever is proposed to Tc39, which is a very small subset of what a language can be. Just look to kotlin, swift and even recent versions of c# for inspiration. Languages have evolved a lot in the past few years and TS hasn't moved much in terms of non-type features. But type features dissapear at compile time and don't add any value at runtime. So I think excessive type-safety is overrated, moderate type-safety is good. As one wise medium user once said to me, "a sound type system does not maketh a language". TS is not forward looking anymore, it's only making what works in js typesafe. I think that was important for full adoption. But I don't think it should be the only goal. I think it should provide a superset of js.Regarding this particular mechanic, I mean we have a flag that says "allowESModuleInterop" that transforms imports. So I don't see why adding another flag, "suffixESImportsOrWhatever" that transforms imports in a slightly different way would be too much of a stretch to imagine. Whether it can accurately do that though, while remapping things like index imports, I don't know. I think this whole add-
.js-at-the-end fiasco is completely unnecessary trouble. Why did we go from nice imports to ugly imports again? Sounds like a step back for mankind.I'm solutions-oriented though, most times the fastest path to a solution is to write your own code, like below.
Solution
Problem: I want TypeScript to automatically add
.jsextensions to my imports.In cases like this I use ttypescript, it basically plugs into the compiler api and lets you run arbitary code as it visits nodes in your AST. Scroll down the readme and you will find a plugin that does exactly what you want, i.e. add
.jsextensions to your relative imports.Also, I wrote this hacky but working script to do a one time migration of my code base.
import glob from 'glob' import { readFileSync, writeFileSync } from 'fs' glob.glob('src/**/*.ts', (err, matches) => matches.forEach(match => writeFileSync( match, readFileSync(match, { encoding: 'utf-8' }) .split('\n') .map(l => l.startsWith('import') && l.includes("'.") && !l.endsWith(".js'") ? l.replace(/'$/, ".js'") : l, ) .join('\n'), { encoding: 'utf-8' }, ), ), )
(js is so expressive these days, you can do all that in a single statement.)
Also, shoutout to tst-reflect which through
ttypescriptgives much better reflection than anything else we have "officially" atm.Reacted by Lewis, Homa Wong, Christian Stewart, referenz, André Posso, staplespeter, Eli and Marcelo SerpaReacted by Daniel Rosenwasser44 remaining items
this was mostly a troll post to give people a place to vent.
Lewis (@ctjlewis) while it may be a troll post in your opinion, it's about a problem real people have in real life in real projects, working for real corps, earning their very real money and having to spend so much time working around a problem introduced by convention and definition without any technical basis, which is maddening. For all practical purposes and to lessen everyone's burden, so much could be done here. And just in the recent weeks we've seen actual improvements on different sides of this problem.
Yes, not everything is on Microsoft, but TS is a standard in the industry, and having no solution here - at least a documented way of getting things done - is a problem. It's not being addressed, which is a no-go for such a widely used tech, and many workarounds have sprouted in the community which points to the problem being a pain-point.
Hence, closing this post will just lead to the problem popping up somewhere else in another issue, possibly with similar discussions to what we already have here. Something I'd like to see is an actionable plan to resolve this issue. Which doesn't mean that the MS team should just do whatever internet randoms tell them to, but decide on some steps to solve the problem in a way they see fit. Which at least should include a chapter in their tutorials addressing it and showing a recipe for common scenarios, like making a lib for different runtimes and module systems. A best-practice we can just copy and be done with it.
TypeScript is an entirely different programming language from JavaScript.
guest271314 yes, TS is a different language than JS. They are still both based on ECMA-262 and TS has the explicit goal of supporting simple transpilation to JS and adding to it. See the official landing page for reference: "TypeScript is a strongly typed programming language that builds on JavaScript." At the same time, TS is an industry standard, and should try to play nice with other parts of the ecosystem. No one cares about perfect integration, however simple recipes on how to deal with common scenarios are a baseline.
Reacted by Lewis, guest271314 and Homa Wongfor me there are two solutions. Enable the transformer plugins, (like ttypescript does) or add somethink like I did in my pull: #47436
They are still both based on ECMA-262 and TS has the explicit goal of supporting simple transpilation to JS and adding to it.
ECMA-262 has some omissions. Observable when we dive in to the minutae.
Take Deno as an example. Deno explicitly supports TypeScript. No need for
ts-node. If you are in to TypeScript you can run TypeScript out of the box on the command line or in.tsscripts.Now, let's examine one impact of supporting static TypeScript programming in the dynamic JavaScript programming language that I just happened to encounter while creating a non-Node.js specific version of
wbn-signNPM package. Dynamic import module is not found when created after the application started. #20945 linking to Do not permission prompt for statically analyzable dynamic imports, specifically this commentI also used to think the same, but it's essentially a given due to how TypeScript supports typings for statically analyzable dynamic imports. Stemming from there it becomes a requirement to preload such imports, which results in things like
write("foo.ts"); await import("./foo.ts")being quirky. And it follows that the runtime net access doesn't occur so it shouldn't prompt.Basically this ship has sailed from the perspective of build tools / static analysis. At this point I think it's viable to carry forward with these behaviours and suggest that users do
const s = "./foo.ts"; await import(s)where they want to opt out.The impact of that decision, per Deno authors based on the internal decision to statically compile dynamic ECMA-262
import()results in the technical fact that we can make Deno'simport()consistently throw - wherenode,bundo not throw under the same conditions Deno dynamic import("./exports") throws module not found for "exports.js" dynamically created in the script// Deno dynamic import("./exports") throws module not found for "exports.js" dynamically created in the script // dynamic_import_always_throws.js // References: https://www.reddit.com/r/Deno/comments/18unb03/comment/kfsszsw/ https://github.com/denoland/deno/issues/20945 // Usage: // deno run -A dynamic_import_always_throws.js // bun run dynamic_import_always_throws.js // node --experimental-default-type=module dynamic_import_always_throws.js import { open, unlink } from "node:fs/promises"; const runtime = navigator.userAgent; const encoder = new TextEncoder(); try { const script = `export default 1;`; // deno if (runtime.includes("Deno")) { await Deno.writeFile("exports.js", encoder.encode(script)); } // node if (runtime.includes("Node")) { const dynamic = await open("./exports.js", "w"); await dynamic.write(script); await dynamic.close(); } // bun if (runtime.includes("Bun")) { await Bun.write("exports.js", encoder.encode(script)); } const { default: module } = await import("./exports.js"); // Raw string specifier console.log({ module }); console.log({ runtime }); } catch (e) { console.log("Deno always throws."); console.log({ runtime }); console.trace(); console.log(e.stack); } finally { console.log("Finally"); // node, bun if (runtime.includes("Node") || runtime.includes("Bun")) { await unlink("./exports.js"); } // deno else if (runtime.includes("Deno")) { await Deno.remove("./exports.js"); } }Now, let's examine the ECMA-262 language for dynamic
import()re specifiers. I don't see anywhere here [13.3.10.1 Runtime Semantics: Evaluation
ImportCall : import ( AssignmentExpression or 13.3.10.1.1 ContinueDynamicImport ( promiseCapability, moduleCompletion ) in the steps at where dynamicimport("./path")inside a running script should throw, per the specification. Some might disagree and claim that runtimes are free to do or not do whatever they want for dynamicimport()specifiers.Deno is only doing this, per the above Deno issue, in order to accomodate TypeScript static compilation for Deno-specific bahaviours. The effecttive result is Deno's implementation of ECMA-262 dynamic
import()is not dynamic.Deno does support WICG Import Maps. Node.js doesn't. Import Maps won't help when we are deliberately running dynamic
import()inside an active script.So TypeScript folks have to reconcile that you can't have a completely static scripting language and support dynamic scripting.
Since TypeScript folks are interested solely in static scripting, bring WICG Import Maps into TypeScript, instead of some custom TypeScript solution that departs further from the dynamic JavaScript programming language.
I'll leave it to the reader to determine whether or not Deno is in conformance with ECMA-262.
Reacted by Arend van Beelen jr. and Homa WongReacted by Luca CasonatoThe first bullet point of https://tc39.es/ecma262/multipage/ecmascript-language-scripts-and-modules.html#sec-HostLoadImportedModule ("or a throw completion") is what makes Deno compliant. ECMA-262 doesn't have a concept of URLs, paths, and file systems at all: it's einteirely implementation-defined.
Nicolò Ribaudo (@nicolo-ribaudo)
The first bullet point of https://tc39.es/ecma262/multipage/ecmascript-language-scripts-and-modules.html#sec-HostLoadImportedModule ("or a throw completion") is what makes Deno compliant.
So that necessarily means in this case Deno behaviour that throws and everybody else's implementation other than Deno that does not throw for raw string specifiers are both compliant?
See what I mean about the vagueness of ECMA-262 in this case?
So that necessarily means in this case Deno behaviour that throws and everybody else's implementation other than Deno that does not throw for raw string specifiers are both compliant?
Yes
See what I mean about the vagueness of ECMA-262 in this case?
And there is a good reason for that: ECMAScript is designed to work everywhere, while I/O is platform-specific. The only I/O that ECMA-262 is aware of is datetime and randomness.
ECMAScript can run on platforms that use URLs, that use filesystem paths, or even on platforms that don't have a concept of a hierarchical filesystem. For this reason, what's the meaning of the string you pass to imports (both static and dynamic) is entirely platform-defined, and individual platforms have to decide what's the meaning of that string and if it corresponds to a module or not.
Yes
I have never observed any specification, standard where throwing is optional, at implemenmter discretion. Where one implementer deliberately throws in one case, other implementers do no throw and bother are conformant to the specification.
Never seen that on any specification for an actual build plan, either. Where having a 10 foot retaining wall is optional. Never seen that in law, either. Where a statute or administrative regulation has the legislative or administrative intent to fine one person, not fine another person for the same conduct or activity.
Nonetheless people will no doubt argue that one implementer can throw for
impoirt("./exports.js")another implementer of the same specification can not throw forimport("./exports.js")- and both are equally comformant to the same specification language.And there is a good reason for that: ECMAScript is designed to work everywhere, while I/O is platform-specific. The only I/O that ECMA-262 is aware of is datetime and randomness.
I actually think that is an omission in ECMA-262. For every JavaScript engine and/or runtime that does implement I/O, each implementatiuon is completely different. There is zero (0) compatibility or interoperability between
readline()of V8'sd8and SpiderMonkey'sjs, nor between Deno, Node.js, QuickJS, txiki.js, et al. https://github.com/guest271314/NativeMessagingHosts - because reading standard input stream and writing to standard output stream, and handling standard error are not specified by ECMA-262.We have all sorts of exotic objects, realms, signals, dynamic
import()that can throw and not throw and both be argued to be conformant to the same language, no basic STDIO written out. That's a huge omission for a general programming language.Nobody seems to notice because people wind up huddling in their own little JavaScript corners, entertaining preferences, shouting whose package registry is bigger, whose been around the longest, and so forth. But try to run the same code in multiple JavaScript engines or runtimes and see what happens. It takes some work to make that so https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_host.js. When it's a simple matter to specify.
But what difference does it make given since we've opened the books any implementation can throw or not throw for any section of ECMA-262. At least the Chromium folks confessed their implementation of
MediaStreamTrackof kind audio is not compliant with W3C Media Capture and Streams, even if they havn't fixed that, yet. It generally takes a few years for Chromium folks to fix stuff, 6 years to get rid of censoringwebkitSpeechRecognition(); 5 years to capture system audio on Linux.TBH if I were in to TypeScript developement and I would modify or abandon that claim that TypeScript is ECMA-262 compliant, or follows the specification. The specification is clearly meaningless when people can interpret throwing and not throwing an error to be equally specification conformant.
Further, it's like Bun claiming it somehow is tryinmg to implement Node.js infrastructure, though we can't full-duplex stream using WHATWG Fwetch in Bun because there's no HTTP/2 support.
Just do your own thing TypeScript folks, without clinging to ECMA-262. Everybody else does their own thing and massages compliance out of throwing an error or not throwing an error.
I'll note this, I have seen such ambiguous language in statues and administrative regulations in the domain of law, re how the preceding or following words can be interpreted by jurists. It's called a term of art in law, and if you ever see this language you best pay careful attention because it generally means the legislature ran out of time, or knows there's some bs going on:
"Notwithstanding any provision to the contrary"
Anyway, good luck trying to be in conformance with ECMA-262, statically analyzing dynamic modules in TypeScript world. Cheers.
Add least adding a setting (wich default is false), like 'addFileExtensionToImportsMissingThemIfTheyAreResolvedWith' would not harm anybody. If you don't need it, keep it false
Add least adding a setting (wich default is false), like 'addFileExtensionToImportsMissingThemIfTheyAreResolvedWith' would not harm anybody. If you don't need it, keep it false
We already have that with WICG Import Maps. That ain't ECMA-262, but neither is WHATWG Fetch or Streams.
<script type="importmap"> { "imports": { "test/file": "./x.json", "test/sub/file": "./z.json" } } </script> <script type="module"> import x from "test/file" with { type: "json" }; import z from "test/sub/file" with { type: "json" }; console.log( new TextDecoder().decode(new Uint8Array([x, z])), import.meta.resolve("test/file"), import.meta.resolve("test/sub/file") ); </script>The dynamic
import()part I see no solution for in TypeScript. TypeScript folks have to massage "we can throw here if we want to so we can statically analyze dynamic modules" into the mix to support the philosophy of statically analyzing everything. Well, you can be static and dynamic at the same time. That's physically impossible. You'll have to make a decision on that. I suspect you'll adhere to Deno's interpretation that, "we can throw here, there's ambiguity or room to do that so we can support TypeScript".- Is this still a problem in practice? I don't remember all the subtleties of this madness, but what I know is that now if I put `module: "commonjs"` and whatever `target` I want in my `tsconfig.json` then I can use relative `import`s without extension in my code and the raw output of `tsc` runs just fine under node. Am I missing something important here? EDIT: Ok I re-familiarized me with the issue. My `tsc` emitted code doesn't use `import`s that's why it works. Are we actually missing out on important stuff by not using `import`s in the JS code?…On Mon, 1 Jul 2024, 15:53 guest271314, ***@***.***> wrote: Add least adding a setting (wich default is false), like 'addFileExtensionToImportsMissingThemIfTheyAreResolvedWith' would not harm anybody. If you don't need it, keep it false We already have that with WICG Import Maps. That ain't ECMA-262, but neither is WHATWG Fetch or Streams. <script type="importmap"> { "imports": { "test/file": "./x.json", "test/sub/file": "./z.json" } } </script> <script type="module"> import x from "test/file" with { type: "json" }; import z from "test/sub/file" with { type: "json" }; console.log( new TextDecoder().decode(new Uint8Array([x, z])), import.meta.resolve("test/file"), import.meta.resolve("test/sub/file") ); </script> The *dynamic* import() part I see no solution for in TypeScript. TypeScript folks have to massage "we can throw here if we want to so we can statically analyze *dynamic* modules" into the mix to support the philosophy of statically analyzing everything. Well, you can be static and dynamic at the same time. That's physically impossible. You'll have to make a decision on that. I suspect you'll adhere to Deno's interpretation that, "we can throw here, there's ambiguity or room to do that so we can support TypeScript". — Reply to this email directly, view it on GitHub <#50501 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALESEZ6TJOS4PGJNS4TZ73ZKFNPBAVCNFSM574CX7O2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMRQGAZDEMJUGQZA> . You are receiving this because you were mentioned.Message ID: ***@***.***>
Are we actually missing out on important stuff by not using
imports in the JS code?CommonJS is not Ecmascript Modules as defined in ECMA-262.
Node.js default loader is CommonJS. There are dozens of JavaScript runtimes that don't use CommonJS.
I'll point out that Bun supports TypeScript out of the box and does not throw as Deno does for dynamic
import()with raw string specifier.Out of curiosity, why are we discussing some Deno-specific behavior here and not in the Deno repo? Especially if Bun shows that this is not a restriction from TypeScript.
Out of curiosity, why are we discussing some Deno-specific behavior here and not in the Deno repo? Especially if Bun shows that this is not a restriction from TypeScript.
Well, why does
tsconfig.jsonhave amodule: "commonjs"option if TypeScript is allegedly ECMA-262 conformant?CommonJS is not specified in ECMA-262.
Out of curiosity, why are we discussing some Deno-specific behavior here and not in the Deno repo? Especially if Bun shows that this is not a restriction from TypeScript.
Because for Deno TypeScript is a first-class citizen. And Deno has bent the interpretation of dynamic
imort()in ECMA-262 to satisfay TypeScript constituents.TypeScript has massaged CommonJS into its configuration to satisfy Node.js constituents.
So there's exceptions to rules happening everywhere here, from a neutral perspective.
I test and break multiple JavaScript engines runtimes at the same time without entertaining a preference for any. So I see what's going on.
Bug Report
This bug report will show that TypeScript is no longer an ECMAScript subset as of ES2015 due to its refusal to support import specifier rewrites for contrived and misplaced reasons.
Preface
This issue is related to previously reported issues, but expands on them and clarifies the scope of the problem.
This team, mostly Andrew and Ryan, have repeatedly shut down these issues in the past. I ask that this issue not be closed on a kneejerk reaction because I intend to speak with others at Microsoft about it.
TS is designed to be an ES superset
TypeScript doubtless has many design goals, and I understand how hard it is to balance all of them. However, undeniably, its core design goal - what TypeScript exists to do - is stated in this repository's description and also throughout the TS docs:
We should all agree that TS, first and foremost, is supposed to be an ECMAScript superset. There is no amount of mental gymnastics that gets us out of this. We will proceed with this in mind as we cover the scope of this issue and some of the technical excuses that have been offered to justify not addressing the problem.
What is a superset?
Originally a set theory concept, in terms of language it was best stated by Sebastian Hungerecker (@sepp2k) in this StackOverflow answer:
To formalize this, Sebastian offers the following definition:
We will not need to address the semantic subset issue as TS fails the first test for arbitrarily many input programs1, which we will get into.
For convenience, and to emphasize the point that our subset programs are actually emitted JS versions of the input TS program, we will refer to a program written in the superset language as "program
T" and the corresponding output in the subset language as "programT'".Additionally, because TS output depends on your compiler configuration, we will consider TypeScript to pass the semantic test if there is some configuration for which you can transform a valid
TintoT'.1 This is something this team is already well aware of, and have simply chosen to ignore. In my judgment, the only possible explanation for violating the core purpose of this technology for such a large portion of its lifecycle is organizational deficiency, and not any practical technical reason.
Is TypeScript a JS superset?
Sometimes. Let's walk through some examples (with approximated and simplified, not literal, output):
✅ Program
T✅ Program
T'The input program is valid and the output program is valid. The runtime behavior of
TandT'are identical. This example passes both the syntactic and semantic superset tests.✅ Program
T(using relative extensionless TS import)❌ Program
T'(TS-style imports left untransformed)The input program is valid, but there is no configuration for which you will get a valid ECMAScript module output. The output program
T'will throw according to the ES spec. This example fails the syntactic and semantic superset tests. Our perfectly valid TypeScript program cannot be transformed by the compiler into a valid ECMAScript program, and this is easily shown.While we can construct a different input program for which we will receive valid output (
import { a } from "./a.js"), making that suggestion is proof positive that TS is no longer an ES superset, and there are infinitely many valid input programs which will not emit valid output. Nothing about this is subjective or within the realm of personal judgment.Bearing this in mind, the last version of ES for which TypeScript was a superset would be ES5 (2009), before the introduction of ES modules.
Why does this matter?
It matters because our industry has struggled to adopt ESM for a variety of reasons, largely related to interop with downstream CJS consumers and also integration with TypeScript. It should be fairly obvious the damage that is done to the ecosystem when the TypeScript compiler very literally cannot generate valid ESM programs without special syntax.
The idea of having to use a special syntax to achieve a correct output program is inherently antithetical to the "TS is an ES superset" principle on which TypeScript is based. It takes a severe amount of technical dysfunction to turn a blind eye to this for years and years, and a serious amount of contempt for users and software quality to continually shut down user feedback on this issue.
Inadequacy of existing solutions and team response to user feedback
I have clarified several times that I am chalking this up to organizational dysfunction and attempting to kick it up the chain myself, as many many many users have raised this issue in isolation only to be shut down and/or directly gaslit by members of this team.
Let's go over some of these issues. I will start with the closest suggestion to a fix currently under consideration, and then go through the rest in chronological order.
(!) 2020: allow voluntary .ts suffix for import paths (#37582) (by Tim Reichen (@timreichen))
I will start with this suggestion as it comes the closest to fixing the problem, while also offering a two-birds-one-stone benefit with regard to Deno, Bun, etc. support:
As specified, this partially solves our problem. Because
import "./a.ts"could only ever refer to a TypeScript program, you cannot conceive of any input.tsimport statement which will not correctly build to a.jsimport and execute equivalently at runtime.However:
Because it is apparently "core design goal" that the imports never get rewritten, the most important part of this suggestion cannot be honored.
(Very important) This demands you rewrite your existing, valid
import "./a"statements to./a.ts, and your valid./aimports still do not build to valid ES import statements.Thus, however we might choose to emit the statements from input, we must either:
TandT'to parity.The current approach by this team has literally just been to do nothing and deny that there is a conflict here. This conflict and its relationship to TS as an ES superset is the core focus of this issue.
2017: TypeScript and <script type="module"></script> (#13422) (by Cyrille Tuzi (@cyrilletuzi))
Because the problem is, again, that TypeScript ESM input programs cannot build to valid output, naturally
<script type="module">goes right out the window.Same pattern: User opens thread, years of comments and confusion, denialism from team members, summarily closed.
2017: Provide a way to add the '.js' file extension to the end of module specifiers (#16577) (by Nikos (@quantuminformation))
This issue looks like the others we will see: A user raising the issue that their valid TypeScript program cannot compile to a valid ES program, a large debate between users and various TS maintainers, and eventually the issue being closed, locked, and buried.
As usual, users point out that TS will act like their imports are valid while refusing to resolve them at compile-time, highlighting the tradeoff I explained. Relative extensionless specifiers must either be allowed and transformed, OR disallowed entirely. Doing both only violates the TS superset principle and confuses users because they depend on their output programs being isomorphic with their inputs.
This comment has 50 likes. The parent issue has nearly 300. Minimizing comments from the team were met with a negative reaction as usual.
2020: Compiled JavaScript import is missing file extension (#40878) (by Jim Fisher (@jameshfisher))
This is an interaction, and a restatement of the same problem, which is effectively the perfect example of not just the problem and the common sense solution to it, but also the community's investment in this being solved, and the textbook denialism by maintainers. The bot response announcing the closure as "working as intended" has 65 downvotes.
Mr. Fisher raises the issue that a program containing valid TypeScript imports will not emit a valid corresponding ECMAScript program. As we will see, this issue is met with approval and enthusiasm by people who found it trying to understand why their programs will not build (after several hours, they will presumably put two and two together and realize it's just not possible). OP is then promptly sandbagged by members of this team and the issue is erroneously closed.
After the issue is posted, Ryan quickly responds to this post by minimizing OP's complaint and telling him to kick rocks:
Again, as we've covered, this is not how supersets work. Simply telling someone to rewrite every already-valid TS import statement into a different one which will compile should seem absurd on its face given what we've gone over. This comment received about 20 downvotes.
Mr. Fisher responds noting that he was shocked to find referring to TS source files with
.js(an extremely confusing and hacky solution that is unbecoming of even a temporary workaround, let alone a permanent solution) actually does resolve his problem. He then lays out a few of the many reasons why this is confusing and hacky. His comment received 50 positive reactions.Ryan responds with another heavily-downvoted dismissal of this issue:
You would expect a valid TS import to become a valid JS import because it's supposed to be a superset. It is not a discussion or an issue of perspective; TS claims to be a superset, if it is, the valid imports would remain valid at runtime. Yet they do not, and we have highly experienced and talented engineers effectively lying about what is and is not expected runtime behavior.
RevealedFrom clarifies this at further length, though it should be pretty obvious already why there are several problems with this, least of all being the superset issue:
Ultimately, it is a bug, and we know it is a bug for the reasons we've covered. Ryan responds again with a minimizing and heavily-downvoted statement I would characterize as dishonest at best, and an outright lazy lie at worst:
We have enough background to know this is wrong, though we are starting to get insight into the executive dysfunction that is causing this issue. Core maintainers are making up what the purpose of this technology is as they go along, apparently now privately rejecting that TS is or ever was meant to be a JS superset.
A user in this thread (Matthieu Gindre (@Gin-Quin)) had to actually implement a bundler to work around this, something I also had to do:
And another user (François-Marie de Jouvencel (@djfm)) also:
And the last comment from this thread I will cover, and emphasize, is this one by Mr Felber (Patrick Felber (@PatrickFelber)):
The only people who claim to not see the problem are members of this team. Note that this user's only choice was to give up and stay on CJS - this is what I am referring to when I say that this is likely the most major blockade to ESM adoption under Why does this matter?.
2020: TypeScript cannot emit valid ES modules due to file extension issue (#42151) (by me)
This issue was opened by me to clarify that the compiler cannot emit valid ES modules. It received positive feedback from users, but was naturally accompanied by denialism from Ryan and eventually closed and locked. A comment by Ricky Miller (@richardkazuomiller) points out the ridiculousness of the justification offered for not fixing this issue ("we can't rewrite import specifiers"):
2021: One year later: TypeScript still cannot emit valid ES modules (#47270) (by me)
This is a rephrasing of the original issue, posted so as to remind this team that this problem has not been resolved. The single response from Ryan was amusing given the context of this issue:
You must decide to prioritize a TypeScript "core design goal", but you can only choose one:
Apparently this is not an obvious answer to TS maintainers, though to users it apparently is. Furthermore this issue is an attempt to follow-up explicitly on the second half of Ryan's request to file something concrete going forward, though I imagine this issue will also be summarily buried as this team appears to have only one tool in its toolbox, namely denial for the sake of convenience.
2020: Appending .js extension on relative import statements during Typescript compilation (ES6 modules) (StackOverflow)
Same situation we've seen: User has a valid input program, can't get a valid output program. The answerer prudently notes:
2022:
"module": "node16"should support extension rewriting (#49083) (by Arend van Beelen jr. (@arendjr))This is the most recent issue that covers this problem. Mr. van Beelen is certainly a lot more generous and forgiving with his wording than I am, though he is equally rewarded with stark denialism from Ryan.
We won't even get into the fact that
import "./a"is not an ES2015 "target-legal" import statement, as it's just an egregiously incompetent claim. It received about 60 downvotes.Mr. van Beelen left several more long, thoughtful comments, generating over 100 positive responses, explaining much of the same things that I have covered here, that valid TS programs should become valid JS programs, though he did so with much more forgiving phrasing. I recommend everyone read his comments, and I found them very considerate - likely impractically so.
This is another thread that contains hundreds of comments, hundreds of positive reactions for the sensible position taken by users, and hundreds of negative reactions for this team's excuses. It is impossible to read it all, but it is worth skimming over and noticing that it, like every other issue raised regarding TS's inability to build valid output programs, was summarily closed for the sake of maintainer sanity to the detriment of several million codebases relying on TS.
2022: Add Compileroption to add Extensions to relative imports (#47436) (by Jochen Kühner (@jogibear9988))
A PR to add non-breaking, opt-in compilerOption for adding
.jsextension to extensionless relative TS imports, which received about 30 positive reactions. Of course, it was summarily closed without a real review. No amount of attempts to contribute by the community have been well-received by this team.🔎 Search Terms
🕗 Version & Regression Information
TypeScript was initially released in 2012, and it has not been an ES superset since the release of ES2015, which require imports to have file extensions if they are not named modules.
⏯ Playground Link
Cannot repro in playground, requires multiple entry points. Minimum repro: https://replit.com/@ctjlewis/ts-esm-superset
Throws
ERR_MODULE_NOT_FOUNDbecause./ais not a valid ES import specifier.💻 Code
N/A, see repro and notes in this issue.
🙁 Actual behavior
The valid TS-ESM program
Tbuilds to an invalid programT'.🙂 Expected behavior
The valid TS-ESM program
Tshould build to a valid programT'. Valid TS imports should always compile to valid ES imports for the given target.