VSCode plugin: Syntax highlighting not correctly applied to embedded svelte component in markdown

[wlach]

Describe the bug
Syntax highlighting not correctly applied to embedded svelte component in markdown when using "Svelte for VSCode"

To Reproduce
Steps to reproduce the behavior:

Put the following inside a markdown document:

```svelte
<script>
  import { onMount } from "svelte";

  export let data = undefined;
  let dom_node;

  onMount(() => {
    Plotly.newPlot(dom_node, data, {barmode: 'stack'});
  });
</script>

<div id="plotDiv" bind:this={dom_node}></div>
```

For example a code snippet that is treated in a way you don't expect.

Expected behavior

Expected similar syntax highlighting to what I see when viewing a svelte file with that content.

Actual behaviour

Get somewhat incorrect / inconsistent formatting (see screenshots)

Screenshots
If applicable, add screenshots to help explain your problem.

View for a svelte component (correct):

View for a svelte component embedded in markdown (incorrect):

System (please complete the following information):

• OS: Mac and Windows 10
• IDE: VSCode
• Plugin/Package: "Svelte for VSCode"

Additional context
Add any other context about the problem here.

[jasonlyu123]

Seem like the injection rules here doesn't work in the markdown code block for some reason. We'll see whether that's possible to fix.

[wlach]

I spent a bit of time today trying to debug this. I used this toy example to test:

```svelte
<script>
  var x = 5;
</script>
```

After giving things slightly more unique names, I found out that vscode's textmate parser seems to be "stuck" in this directive when in markdown:

language-tools/packages/svelte-vscode/syntaxes/svelte.tmLanguage.src.yaml

Line 467 in 6e0396c

endCaptures: { 0: { name: punctuation.definition.tag.end.svelte } }

That is to say, when I'm at var inside a script block, it still thinks I'm in meta.tag.start.svelte (renamed to meta.tag.start3.svelte in my local checkout). i.e. the scopes are:

entity.other.attribute-name.svelte
meta.attribute.var.svelte
meta.tag.start3.svelte
meta.script.svelte
meta.embedded.block.svelte
markup.fenced_code.block.markdown
text.html.markdown

What's odd is when I put the cursor at the very end of the script block above (i.e. just after the >), the textmate scopes are:

punctuation.definition.tag.end.svelte
meta.tag.start3.svelte
meta.script.svelte
source.svelte

So it's noticing the end of the block, but somehow not transitioning into the state where it's actually looking at the JavaScript. I assume that's:

language-tools/packages/svelte-vscode/syntaxes/svelte.tmLanguage.src.yaml

Line 18 in 6e0396c

'L:meta.script.svelte meta.lang.javascript - (meta source)':

Why would it be triggering endCaptures but not actually ending the block? It's very strange.

[wlach]

Had the bright idea of using git bisect, which revealed that this used to work (when it was added in July 2020 with #301) but seems to have been broken by #657

[wlach]

@Monkatraz any ideas on ^^^ ?

[wlach]

Apologies for my flailing around. Dug into things more and realized that @jasonlyu123 was correct all along: the injection rules are not being applied in an embedded context. The weird behaviour I'm describing above is only due to the fact that the injection rules are not there, and it's falling back to whatever else is defined in the file.

I'm now pretty sure this is an issue upstream so filed an issue there: microsoft/vscode-textmate#152 -- that has some more details as well as confirmation of the problem described.

It's likely possible to make the grammar not depend on injections (e.g. I got it working for bare script tags without them), though I think it would be less elegant.

[dummdidumm]

Thank you for digging into this!

[wlach]

Based on the response in microsoft/vscode-textmate#152, it looks like we might have to rewrite this not to use injection grammars (at least if we care about fixing this bug).

[Monkatraz]

Doing that will be horribly messy... How frustrating.

[dummdidumm]

At least this comment gave me another possibility on how to better debug grammar matches

[Monkatraz]

Okay I guess I need to do three things sooner or later:

• find a decent way of adding embedded languages that doesn't involve injections
• Make a separate PR for the --css-var attribute highlighting
• Clean up my miniature CSS grammar PR

I've been working with CodeMirror 6 a lot recently and the highlighting there is so much better it's jarring going back to VSCode lol (I should make a Svelte grammar for it)

[Monkatraz]

Although I should add that one alternative is to use a different grammar for markdown, I think one that always uses TypeScript would be a decent compromise. Obviously that has edge cases (what if you're using something nutty like CoffeeScript or Rescript)