Overhaul GML highlighting

[thennothinghappened]

Changes

I've created a proper grammar for GameMaker Language, which supports many of the language's newer features, and goes a lot further than the existing one - which currently just matches a master list of keywords and function names that must be kept up to date.

The work I've done here is a port of my previous PRs to the GameMaker Manual (YoYoGames/GameMaker-Manual#160, YoYoGames/GameMaker-Manual#220), though since they use a bit of a customised fork, required some extra work to then bring upstream. As this PR encompasses the changes I listed in those PRs above I've avoided re-stating the same changes, though if it would be preferable I can bring them over into this PR's description.

Additional to the PRs however, I've also made some further minor fixes and improvements:

• JSDoc is now highlighted, in all forms stated in the GameMaker Manual.
• Multi-line #macros are now parsed properly, though this isn't currently a noticeable change.
• Many obsolete or incorrect keyword entries have been removed, and some new ones have been added.
• The gml.css theme has been updated accordingly with the changes with scopes to more correctly follow the GM IDE's colour scheme.

Comparison

As a visual overview of the changes, here's a before and after!

Before

After

Checklist

• Updated markup tests.
• Updated the changelog at CHANGES.md

[joshgoebel]

This is a huge change and will need a bit of going thru. I'll drop comments.

[joshgoebel]

What is the thinking that this type of string only scopes the string delimiters rather than the entire string?

[thennothinghappened]

I was having a lot of issues getting template literals to behave as I wanted. The issue was that if I simply set scope: "string", then the entire thing would be wrapped with that class, which would then make anything within templates, e.g.,$"{someVariable}" appear string-coloured if not claimed by another expression type (e.g., a number would still look like a number, since it would claim it.)

I might have a look at how some other languages approached this issue as I wasn't happy with this solution regardless, but I recall going down a rabbit hole of greater and greater complexity language-knowledge by the parser to fix this - there's probably an easy solution I missed though :P

[thennothinghappened]

I've come back to this again and came up with a middleground that I think is a lot more reasonable - this has been kept, but I've fixed the issue where string characters were being matched one-by-one which was making a ridiculous number of individual <span>s for each character. This has been fixed, but the string scope is still not applied to the entire span, so that the inner templates are not interfered with by string in themes.

[joshgoebel]

Maybe we'd allow some liberty with the GML theme... not sure... but typically nested items like:

string -> subst -> number

The subst color would override string, and the number color would override subst. If a theme doesn't provide a style for subst that theme is broken and should be fixed. Background can be a real problem, but not a lot of themes do crazy things with backgrounds - is GML one of them?

Still not entire I'm completely following your issue.

[thennothinghappened]

The root of the issue is that identifiers for things like local variables do not have any matching parse rule, and are unhighlighted. I didn't add any "catch all" rule at the bottom for this, though I guess I could throw in such a rule for anything that's a valid identifier with a relevance of 0. As this rule does not currently exist, here's what happens if I let the scopes be nested:

...the result being that these variables become string-coloured.
However, using the setup that's currently in, the issue is resolved:

Another thing going for this to me was that it avoids the issue where a nested scope would inherit things you don't want it to - e.g., if you had a theme which made strings bold, you don't really want the contents of their templates to also become bold with them.

Edit: I also went ahead and tried changing out the beginScope and endScope on templates to just scope. This follows suit with what JS does. While this does look acceptable and I can go this route if preferable, it still seems like a less favourable option as it diverges from the sort of highlighting you get from the GM IDE's semantic highlighting, which treats the contents of $"{in_here}" identically to were that expression written outside of a string template.

[joshgoebel]

Ok, I looked closely at your mode for string templates now - yeah this is not the preferred way - matching string like chunks inside strings. :). Neat trick though.

Isn't this same effect achievable thru pure CSS with just a ~ .string .subst rule that sets the styling (color, bg, bold/italic) back to whatever the theme considers "standard"?

This is the kind of thing I"d prefer to solve at the engine/CSS level rather than encouraging every individual grammar to start adding "hacks" like this to accomplish stylistic goals.

[thennothinghappened]

Totally understand about avoiding these sorts of hacks.

As for the alternative, if I had .subst reset the colour scheme in CSS, it'd also cause the surrounding {} to become an un-highlighted colour. I've taken that idea though and modified it such that .subst resets colours, but the {} is given char.escape instead - would this be an acceptable solution?

With a minor tweak to gml.css, this results in the intended behaviour.

[thennothinghappened]

Thanks for the thorough review! I've addressed everything I could, though there's two left unresolved with comments as I was unsure how to proceed with them so leaving for further feedback.