(XML) Unicode letters should be recognized in element and attribute names

[martin-honnen]

Describe the issue
XML allows Unicode letters as element and attribute names while your xml.js language mode uses a regular expression just checking for ASCII letters A-Z.
That way anyone trying to highlight XML with non-ASCII letters in element or attribute names doesn't get highlighting e.g. in <categoría>producto</categoría> the Spanish word categoría which is a well-formed XML element name is not recognized as such by the regular expression const TAG_NAME_RE = regex.concat(/[A-Z_]/, regex.optional(/[A-Z0-9_.-]*:/), /[A-Z0-9_.-]*/); in https://github.com/highlightjs/highlight.js/blob/main/src/languages/xml.js#L12

Which language seems to have the issue?
XML from https://github.com/highlightjs/highlight.js/blob/main/src/languages/xml.js

Are you using highlight or highlightAuto?

highlight

Sample Code to Reproduce

console.log(hljs.highlight(`
<root>
  <categoría>test</categoría>
  <category>test</category>
</root>`, {language: 'xml'}).value)

Expected behavior

The output for the XML markup <categoría>test</categoría> currently is <categoría>test</categoría> while it should be <span class="hljs-tag">&lt;<span class="hljs-name">categoría</span>&gt;</span>test<span class="hljs-tag">&lt;/<span class="hljs-name">categoría</span>&gt;</span>.

Additional context

https://www.w3.org/TR/xml/#NT-NameStartChar and https://www.w3.org/TR/xml/#NT-NameChar definitions from XML spec. I think it should be possible to fix the regular expressions used in xml.js, either by using ranges of the characters given in the XML spec or, if the Unicode support in JavaScript regular expressions is used, by using e.g. \p{Letter} instead of A-Z.

[martin-honnen]

I am not sure the whole range of Unicode letters or the same range as used in the XML spec is expressible in the four digit \uDDDD format JavaScript supports but a much broader range than the current ASCII A-Z could be implemented with e.g.

nameStartChar = /[A-Z_a-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD]/g

I think.

[joshgoebel]

Could we just use \p{L} for this now instead of that huge list?

https://stackoverflow.com/a/51413159/12430243

[joshgoebel]

If so perhaps lets reopen #3257 with that approach instead...

[martin-honnen]

I will hopefully be able to look into this next week.

[joshgoebel]

Ping. Still interested in pursuing this?

[martin-honnen]

I will give it a try during this week (i.e. until Sunday, May 1st).