CS2 Discussion: Features: Flag for ESNext output from current compiler

[GeoffreyBooth]

This is a proposal for generating ES2015+ syntax from the current compiler. It’s rather simple. We add a flag:

--ecmascript    Output the latest ECMAScript syntax whenever possible.

To implement this, in the compiler itself we can either have if blocks within node classes that generate different output based on whether this flag is set; or we can have a second set of node classes that override the first set if this flag is set.

This flag would only apply to features like await, where we have a PR that implements an ES5 polyfill but others might prefer the await keyword output as is; and features like => that are supported in ECMAScript now and people might prefer to output them as is rather than polyfilling them. Eventually this flag could allow us to automatically output let or const as appropriate, without worrying about breaking backward compatibility. Modules support would be unaffected by this flag, as using the import or export keywords would be opting in to ECMAScript for those lines; there’s no point in making such lines throw an error if --ecmascript is unset. Likewise for generators. With regard to classes, I’m assuming based on #22 that we will likely implement a new keyword that produces ECMAScript classes, leaving the current class keyword alone and allowing projects to contain both types; so those keywords would also be unaffected by this flag.

This flag isn’t meant to foreclose the effort at creating a new compiler; on the contrary, it might pave the way for it. We can implement this flag now, with maybe just one or two features compiled differently, with the list growing over time; but when the new compiler is ready and can handle all the same features that the legacy compiler plus --ecmascript can handle, the --ecmascript flag could trigger opting into the new compiler.

[rattrayalex]

In terms of implementation, couldn't it simply wrap decaffeinate?

[GeoffreyBooth]

In terms of implementation, couldn't it simply wrap decaffeinate?

No. decaffeinate doesn’t convert all code correctly. It’s meant to preserve code including comments and so on. There are things it can’t parse, that it leaves for the developer to convert by hand. See https://github.com/decaffeinate/decaffeinate/blob/master/docs/conversion-guide.md

[GeoffreyBooth]

Update: I just found this comment where @jashkenas proposes a flag for ES6 output. So I guess a flag for this purpose would actually be blessed after all.

[GeoffreyBooth]

There are some questions we would need to hammer out (@lydell?):

• How do we divide the code? The simplest approach would be an if block within the relevant classes in nodes.coffee, but a better one might be to create a separate nodes_ecmascript.coffee that overrides those classes from the first one. And over time as we implement more ECMAScript-output versions of features, nodes_ecmascript.coffee would grow.
• We would also need tests that only run in --ecmascript mode, and vice versa. How would we enable that?

This also ties into the discussion of a new compiler. Basically, with this flag I’m assuming a new compiler is a ways off. Both await and tagged template literals would have a use/need for this flag now, literally this week or next, and I don’t want to hold up those features or build them only as ES5 shimmed versions because we lack this flag or a working new compiler. If and when the new compiler arrives, --ecmascript could be redirected to triggering the new compiler, or it could get its own opt-in flag or new repo/NPM module.

[GeoffreyBooth]

@rattrayalex Decaffeinate might not be an option, but Decaf might be . . .

[greghuc]

If there's an --ecmascript flag, the compiler could be more blunt about erroring. So if someone uses an ESnext-backed feature (like await or tagged template literals) without the flag, the compiler errors and says "you need the --ecmascript flag to use this feature". And there's no polyfill for these features.

From a documentation perspective, the flag is also useful. We put a section at the end of the Readme listing the ESnext-only features which need the flag. And this sets the narrative for what's happening: if/when the compiler switches to just ESnext output, it can be explained in terms of the flag ("flag is now on by default").

[jashkenas]

I think that a more appealing path than a --modern-style flag is to simply say that CoffeeScript 1.x is for ES3-era browsers, and CoffeeScript 2.x+ is for the evergreen browsers of today. Then you can make your breaking changes, release the new compiler, and folks that want to use it, can.

[mrmowgli]

Coffeescript 2 should still end up being coffeescript. I think the problem a lot of these projects run into is that they don't become the defacto compiler. At some point the base coffeescript branch should merge the new changes from ES6, and you just use coffeescript. You've already integrated yield, which is an ES5+ feature. The real question is the best way to ease that transition, or have some procedure for transitioning, much like io.js -> node.js development. io.js is the unstable new feature set, node.js is the production ready version. @jashkenas, is there a reason you have been avoiding the use of flags?

[GeoffreyBooth]

I think CoffeeScript 2.0 is what would be produced by the --modern or --ecmascript flag. The only question is whether developing the ESNext-outputting compiler should happen in a 2.0 branch or in the current branch behind a flag.

The branch approach has been taken by Gulp in their long march toward version 4.0; for the last two years I’ve been putting "gulp": "gulpjs/gulp#4.0", in my package.json. It has its merits while developing in that you’re not looking at two versions of how features should be implemented; either you’re on the 2.0 branch or you’re not. The codebase is simpler.

Where it gets tricky, though, is if we want to add a new feature like await that could be added as both legacy and ESNext versions. The only way to do such a thing is to open two pull requests, one to merge into 2.0 and the other to merge into master. As the PRs would get reviewed, there would be duplication of effort for revisions that need to be applied to both PRs.

We could certainly call the --modern-flagged output “CoffeeScript 2.0,” and eventually release a 2.0.0 module to NPM that has --modern enabled by default. @jashkenas do you really care if this work happens behind a flag versus on a branch? I think this decision might best be made by whoever takes on this work (not that that will be me).

[jashkenas]

@jashkenas do you really care if this work happens behind a flag versus on a branch?

I don't care terribly much per se, but my strong advice would be that having two code generation paths in an already complex codebase will make things that much more difficult to work with and continue to develop.

For development, maintainability, for marketing, and for user friendliness — I think it would be smart to work and release from a 2.0 branch. And to tell people that CoffeeScript 2+ targets evergreen JavaScript.

[JimPanic]

I like that idea. Although as long as we don't introduce anything breaking backwards compatibility, semver kind of dictates to follow along the path of 1.x. (Has CS even been using semver so far?)

But given classes are a top priority, together with getters and setters, this is not too far away either.

[GeoffreyBooth]

@jashkenas I’m not opposed to a new branch, but I think any “new” compiler is likely to include falling back to the “old” one, so we’re going to have a complicated codebase regardless. Like for example maybe class gets handled by the new compiler, and all other nodes get handled by the old. Until the new compiler is feature-complete with the old, we’ll have both littering up the codebase. But I doubt the new compiler would ever be feature-complete; more likely, it would only process features that we want to output differently than the old compiler (like =>) and leave everything else to the current compiler.

This assumes that the new compiler takes a different approach than the old one, for example what we’ve been discussing about ASTs in #25. If CoffeeScript 2.0 outputs ESNext syntax using same string-generation compiler that we have now, then sure, it should be in a new branch and there aren’t two code generation paths.