Description
I’m thinking that maybe the way to handle things like JSX (see #4529) or type annotations is to build in hooks for plugins into CoffeeScript. Specifically, there would be two: preprocessors and postprocessors. We already have one preprocessor already: invertLiterate, the function that separates the comments from the code for Literate CoffeeScript. That function basically just parses each line and adds # before any line that isn’t indented. What if we split that function out into its own module?
When the compiler runs, it would see that the “literate” module is loaded and that it provides a function to run in the preprocessors hook, and so CoffeeScript would run that function before parsing any code, the same way it runs invertLiterate now. The function would take the code string as input and return a modified version of the string as output (along with source maps, if necessary). And likewise for functions provided for postprocessing. Someone could write a plugin that provides a postprocessing hook for transpiling through Babel, for example. Some plugins could provide both types of hooks, for example to remove code before passing it through the compiler and restoring it afterward. Plugins could also specify dependencies, like that they should always run before or after a certain other plugin, so that people could enable multiple plugins to have several functions run in sequence in the same hook.
This opens the floodgates for any extensions to CoffeeScript that people might want to enable on a project-by-project basis, without having to fight to get the modification included into the language itself. This could be the solution for the “Coffee Tags” proposal for adding a JSX-like syntax; or for macros.
The problem to solve would be how plugins could pass non-CoffeeScript code through the compiler. Backticks are currently allowed only where the embedded JS can be an expression; a line like return `<h1>` @title `</h1>` is disallowed. Block comments are even more restricted, allowed only as statements or inside object literal assignments; something like ### String ### name = 'Joe' is also disallowed. Is there a way we could expand the types of places that embedded JavaScript or block comments could be permitted? I understand that such tokens can’t simply be allowed everywhere, since lines like eat food for food in foods when `/* String */` food isnt 'chocolate' cause significant new code generation and therefore it’s ambiguous where the embedded JS /* String */ should go in the resulting output. But perhaps we can increase such tokens’ utility enough to cover most cases that people would want to use them for, for example for JSX tags or type annotations. Such improvements would fix #4464.
An even more ambitious plugin architecture would involve hooks for new types of tokens in the lexer itself, like what the “Coffee Tags” proposal adds. This would spare plugins from having to implement their own lexing and parsing logic. But for the first version of this, I think some way to pass through embedded JavaScript or /* */-style comments would solve several of the most common use cases.