Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enh(xml) Allow Unicode attribute and tag names #3257

Closed
wants to merge 5 commits into from
Closed

enh(xml) Allow Unicode attribute and tag names #3257

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5abdcd1
test case with non-ascii character in XML element name
martin-honnen Jun 24, 2021
686df4f
Fixing TAG_NAME_RE in xml.js to include Unicode ranges the XML spec a…
martin-honnen Jun 25, 2021
9120095
Fixing XML_IDENT_RE in xml.js to also include non-ASCII letters the X…
martin-honnen Jun 25, 2021
931aa55
Some clean-up of explaining comments and removal of previously outcom…
martin-honnen Jun 25, 2021
079105d
Updating CHANGES.md
martin-honnen Jun 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

Grammars:

- fix(xml) allow Unicode letters in XML element and attribute names(#3256)[Martin Honnen][]
- fix(csharp) add missing `catch` keyword (#3251) [Konrad Rudolph][]
- add additional keywords to csp.js (#3244) [Elijah Conners][]
- feat(css) handle css variables syntax (#3239) [Thanos Karagiannis][]
Expand All @@ -18,6 +19,7 @@ Grammars:

[Stel Abrego]: https://github.com/stelcodes
[Josh Goebel]: https://github.com/joshgoebel
[Martin Honnen]: https://github.com/martin-honnen
[Antoine Lambert]: https://github.com/anlambert
[Elijah Conners]: https://github.com/elijahepepe
[Angelika Tyborska]: https://github.com/angelikatyborska
Expand Down
12 changes: 8 additions & 4 deletions src/languages/xml.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,14 @@ Audit: 2020
import * as regex from '../lib/regex.js';

/** @type LanguageFn */
export default function(hljs) {
// Element names can contain letters, digits, hyphens, underscores, and periods
const TAG_NAME_RE = regex.concat(/[A-Z_]/, regex.optional(/[A-Z0-9_.-]*:/), /[A-Z0-9_.-]*/);
const XML_IDENT_RE = /[A-Za-z0-9._:-]+/;
export default function (hljs) {
// XML names start with the following Unicode characters: https://www.w3.org/TR/xml/#NT-NameStartChar
// NAME_START_CHAR = /[A-Z_a-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD]/;
// XML names can have the following additional letters: https://www.w3.org/TR/xml/#NT-NameChar
// OTHER_NAME_CHARS = /[:\-.0-9\u00B7\u0300-\u036F\u203F-\u2040]/;
// Element names start with NAME_START_CHAR followed by optional other Unicode letters, ASCII digits, hyphens, underscores, and periods
const TAG_NAME_RE = regex.concat(/[A-Z_a-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD]/, regex.optional(/[A-Z_a-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD\-.0-9\u00B7\u0300-\u036F\u203F-\u2040]*:/), /[A-Z_a-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD\-.0-9\u00B7\u0300-\u036F\u203F-\u2040]*/);;
const XML_IDENT_RE = /[A-Z_a-z:\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD\-.0-9\u00B7\u0300-\u036F\u203F-\u2040]+/;
const XML_ENTITIES = {
className: 'symbol',
begin: /&[a-z]+;|&#[0-9]+;|&#x[a-f0-9]+;/
Expand Down
1 change: 1 addition & 0 deletions test/markup/xml/non-ascii-element-name.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<span class="hljs-tag">&lt;<span class="hljs-name">categor�a</span>&gt;</span>producto<span class="hljs-tag">&lt;/<span class="hljs-name">categor�a</span>&gt;</span>
1 change: 1 addition & 0 deletions test/markup/xml/non-ascii-element-name.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<categor�a>producto</categor�a>