[Proposal] Run js expressions in markup template through Svelte script preprocessor code

[swyxio]

Is your feature request related to a problem? Please describe.
i would like to use babel/typescript syntax inside of Svelte markup template code.

For example, let's say i have the babel optional chaining enabled:

// rollup.config.js
    svelte({
	  // ...
      preprocess: {
        script: ({ content }) => {
          return require("@babel/core").transform(content, {
            plugins: ["@babel/plugin-proposal-optional-chaining"],
          });
        },
      },
    }),

This lets me use new JS syntax in my script tag:

<script>
  let foo = {
		bar: {
			baz: true
		}
	}
  let sub = foo?.ban?.baz
</script>

<main>
  <h1>Hello {sub}!</h1>
</main>

this is great! however we try to move it down and Svelte complains:

<script>
  let foo = {
		bar: {
			baz: true
		}
	}
</script>

<main>
  <h1>Hello {foo?.ban?.baz}!</h1>
  <!-- uh oh -->
</main>

Error:

[!] (plugin svelte) ParseError: Unexpected token
src/App.svelte
 6: 
 7: <main>
 8:   <h1>Hello {foo?.ban?.baz}!</h1>
                     ^
 9: </main>
ParseError: Unexpected token
    at error (/Users/swyx/Desktop/Work/testbed/trybabelrecipe/svelte-app/node_modules/svelte/src/compiler/utils/error.ts:25:16)
    at Parser$1.error (/Users/swyx/Desktop/Work/testbed/trybabelrecipe/svelte-app/node_modules/svelte/src/compiler/parse/index.ts:96:3)

This is somewhat of a break of the mental model, since Svelte should accept the same kinds of JS inside templates as it does inside script tags. You can imagine other good usecases for this, e.g. {#if data?.foo?.bar}

in order to fix this, i would have to hook into a markup preprocessor, and then parse the template contents to sift out the js expressions, and then transpile it, and then stitch the markup back together. seems like repeat work to what Svelte already does anyway.

Describe the solution you'd like

Svelte should reuse the script preprocessor for in-template js expressions, as well as for <script> tags.

Describe alternatives you've considered

no change

How important is this feature to you?

it's a nice to have. i dont have any proof, but i suspect this might help the TypeScript effort too in case any assertions are needed (this is rare, though; more likely the "new syntax" usecase is more important than the "need to specify types in expressions" usecase)

[swyxio]

tagging @pngwn for any other comments, and maybe @Conduitry and @tanhauhau. happy to work on this if you think this is a good idea. main tradeoff i can think off is that it might make svelte template compilation a tiny bit slower, but i dont believe it will be noticeable.

[pngwn]

I feel like running script preprocessors on every JS expression in the template would slow down compilation considerably but have no data to back that up.

The other option here would be to expose an api to parse the template without passing the JS expressions to acorn yet. At least then you could write a relatively simple markup preprocessor to handle this case.

[Conduitry]

The main technical problem here is that it involves Svelte identifying template expressions written in a language that it cannot parse. The beginnings of template expressions are found via the { but the end is found by telling Acorn 'okay parse as much as you can here as an expression' and then Svelte makes sure there's a } after where Acorn says it parsed until.

As a sort-of solution, the preprocessor could just count opening an closing braces, but this would be thrown off by, say, expressions containing strings containing mismatched braces. A proper solution to this seems more to be to create a new callback that will be called, one at a time, with the start of each expression, and returns the preprocessed code and how many characters it ate, or else throws an exception. I don't have any opinions yet on what this API should look like or what it should be a part of.

[Conduitry]

If completely avoiding duplicating Svelte parser logic in a preprocessor is not a goal, this all could be fairly straightforwardly implemented in userland in a template preprocessor. The template preprocessor runs before the script or style preprocessors, and the initial intention was that it would do something like convert Pug to HTML, but there's nothing stopping it from transforming any part of the component that it wants. The script and style preprocessors are pretty much sugar on top of the template preprocessor, in that they extract the script or style tags and then operate only on that, leaving the rest of the component unchanged.

[swyxio]

indeed we discussed a userland preprocessor solution - but it seemed less than ideal bc it would basically duplicate work that Svelte/Acorn is already doing, and involve wiring up babel twice (once for script, once for template)

as a Svelte user, the mental model for script preprocessor is "ok this thing works on everything inside the <script> tag". but that's not the only place that Javascript appears in a Svelte component.

re: the speed - the syntax transpiles here are going to be very light. will have to benchmark to know for sure ofc but we're definitely not going thru the whole es5 dance here.

I dont know much about the Svelte-Acorn handoff, so this wrinkle with the parsing is interesting. happy to explore that new callback, but i wonder if it should just be the same callback that the preprocessor is

[chopfitzroy]

Wanted to chime in here as no googling found this issue (attributing this to the age of the issue) which lead to me creating the above duplicate.

To distill what I was trying to say in the above issue I think irrespective of the decision made here it would be good to have some formal documentation on this (also mentioned in #3388) to guide new users and help mitigate wasted effort in trying to set this up when it is currently not possible.

[swyxio]

based on what conduitry said, it sounds like some R&D is needed on improving that svelte-acorn parsing. having different systems take care of the { and the } seems a little brittle? idk

[tanhauhau]

just throwing some of my thoughts over here, the current preprocessor is more of a string based preprocessor, doing string replacements before letting svelte compiles it.
this has few implications:

• we dont want to parse the code before parsing the code, therefore, we use regex to quickly match out the <script> and <style> tag. however matching { } brackets and is hard.
• we discard ast and sourcemap from the preprocessor, for example, if using a typescript preprocessor, we use the generated js code and discard away typescript ast, and reparse the js code in svelte. if typescript AST is estree compliant, why spend extra effort on parsing them again?
  • and this could be make using sourcemap from preprocessor easier

so i would like to propose that, maybe instead of preprocessor, we can have a parser plugin.

we can have a default acorn plugin to help parse JS, but also allow user to provide custom JS parser, as long as they are estree compliant.
same idea can go to css too.

[swyxio]

i didnt quite understand the difference between preprocessor or parser plugin, but i also didnt really understand the whole post haha. if this seems like the better idea, where is a good place to start?

also...what does "same idea can go to css too" mean? how would this help?

[tanhauhau]

oh i think i didnt explain it clearly..

i think what i was trying to say is that currently it's hard to make its hard to run js expression in markup through preprocessor as it is run before the svelte parsing.

so maybe an alternative is to provide a way to tap into the parsing, to allow user to provide custom callback to parse JS expression

[Conduitry]

If we did integrate this into the parsing by making it an option to the compiler (rather than handling it in an earlier preprocessing step), we'd need to enforce that those plugins are synchronous. And then we'd have two different ways of doing what would seem to users to be very similar things, and each would have different limitations, which sounds confusing.