Skip to content

feat: ecmascript 13 unicode properties #1084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Dec 11, 2021
Merged

feat: ecmascript 13 unicode properties #1084

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
eb901a4
feat: ecmascript 13 unicode properties
dnalborczyk Dec 11, 2021
b37a23d
fix: remove duplicate unicode script values
dnalborczyk Dec 11, 2021
2bfbfc1
refactor: use const
dnalborczyk Dec 11, 2021
b31ad21
refactor: build unicode data
dnalborczyk Dec 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion acorn/src/regexp.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ export class RegExpValidationState {
constructor(parser) {
this.parser = parser
this.validFlags = `gim${parser.options.ecmaVersion >= 6 ? "uy" : ""}${parser.options.ecmaVersion >= 9 ? "s" : ""}${parser.options.ecmaVersion >= 13 ? "d" : ""}`
this.unicodeProperties = UNICODE_PROPERTY_VALUES[parser.options.ecmaVersion >= 12 ? 12 : parser.options.ecmaVersion]
this.unicodeProperties = UNICODE_PROPERTY_VALUES[parser.options.ecmaVersion >= 13 ? 13 : parser.options.ecmaVersion]
this.source = ""
this.flags = ""
this.start = 0
Expand Down
20 changes: 12 additions & 8 deletions acorn/src/unicode-property-data.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,31 +9,35 @@ const ecma9BinaryProperties = "ASCII ASCII_Hex_Digit AHex Alphabetic Alpha Any A
const ecma10BinaryProperties = ecma9BinaryProperties + " Extended_Pictographic"
const ecma11BinaryProperties = ecma10BinaryProperties
const ecma12BinaryProperties = ecma11BinaryProperties + " EBase EComp EMod EPres ExtPict"
const ecma13BinaryProperties = ecma12BinaryProperties
const unicodeBinaryProperties = {
9: ecma9BinaryProperties,
10: ecma10BinaryProperties,
11: ecma11BinaryProperties,
12: ecma12BinaryProperties
12: ecma12BinaryProperties,
13: ecma13BinaryProperties
}

// #table-unicode-general-category-values
const unicodeGeneralCategoryValues = "Cased_Letter LC Close_Punctuation Pe Connector_Punctuation Pc Control Cc cntrl Currency_Symbol Sc Dash_Punctuation Pd Decimal_Number Nd digit Enclosing_Mark Me Final_Punctuation Pf Format Cf Initial_Punctuation Pi Letter L Letter_Number Nl Line_Separator Zl Lowercase_Letter Ll Mark M Combining_Mark Math_Symbol Sm Modifier_Letter Lm Modifier_Symbol Sk Nonspacing_Mark Mn Number N Open_Punctuation Ps Other C Other_Letter Lo Other_Number No Other_Punctuation Po Other_Symbol So Paragraph_Separator Zp Private_Use Co Punctuation P punct Separator Z Space_Separator Zs Spacing_Mark Mc Surrogate Cs Symbol S Titlecase_Letter Lt Unassigned Cn Uppercase_Letter Lu"

// #table-unicode-script-values
const ecma9ScriptValues = "Adlam Adlm Ahom Ahom Anatolian_Hieroglyphs Hluw Arabic Arab Armenian Armn Avestan Avst Balinese Bali Bamum Bamu Bassa_Vah Bass Batak Batk Bengali Beng Bhaiksuki Bhks Bopomofo Bopo Brahmi Brah Braille Brai Buginese Bugi Buhid Buhd Canadian_Aboriginal Cans Carian Cari Caucasian_Albanian Aghb Chakma Cakm Cham Cham Cherokee Cher Common Zyyy Coptic Copt Qaac Cuneiform Xsux Cypriot Cprt Cyrillic Cyrl Deseret Dsrt Devanagari Deva Duployan Dupl Egyptian_Hieroglyphs Egyp Elbasan Elba Ethiopic Ethi Georgian Geor Glagolitic Glag Gothic Goth Grantha Gran Greek Grek Gujarati Gujr Gurmukhi Guru Han Hani Hangul Hang Hanunoo Hano Hatran Hatr Hebrew Hebr Hiragana Hira Imperial_Aramaic Armi Inherited Zinh Qaai Inscriptional_Pahlavi Phli Inscriptional_Parthian Prti Javanese Java Kaithi Kthi Kannada Knda Katakana Kana Kayah_Li Kali Kharoshthi Khar Khmer Khmr Khojki Khoj Khudawadi Sind Lao Laoo Latin Latn Lepcha Lepc Limbu Limb Linear_A Lina Linear_B Linb Lisu Lisu Lycian Lyci Lydian Lydi Mahajani Mahj Malayalam Mlym Mandaic Mand Manichaean Mani Marchen Marc Masaram_Gondi Gonm Meetei_Mayek Mtei Mende_Kikakui Mend Meroitic_Cursive Merc Meroitic_Hieroglyphs Mero Miao Plrd Modi Modi Mongolian Mong Mro Mroo Multani Mult Myanmar Mymr Nabataean Nbat New_Tai_Lue Talu Newa Newa Nko Nkoo Nushu Nshu Ogham Ogam Ol_Chiki Olck Old_Hungarian Hung Old_Italic Ital Old_North_Arabian Narb Old_Permic Perm Old_Persian Xpeo Old_South_Arabian Sarb Old_Turkic Orkh Oriya Orya Osage Osge Osmanya Osma Pahawh_Hmong Hmng Palmyrene Palm Pau_Cin_Hau Pauc Phags_Pa Phag Phoenician Phnx Psalter_Pahlavi Phlp Rejang Rjng Runic Runr Samaritan Samr Saurashtra Saur Sharada Shrd Shavian Shaw Siddham Sidd SignWriting Sgnw Sinhala Sinh Sora_Sompeng Sora Soyombo Soyo Sundanese Sund Syloti_Nagri Sylo Syriac Syrc Tagalog Tglg Tagbanwa Tagb Tai_Le Tale Tai_Tham Lana Tai_Viet Tavt Takri Takr Tamil Taml Tangut Tang Telugu Telu Thaana Thaa Thai Thai Tibetan Tibt Tifinagh Tfng Tirhuta Tirh Ugaritic Ugar Vai Vaii Warang_Citi Wara Yi Yiii Zanabazar_Square Zanb"
const ecma9ScriptValues = "Adlam Adlm Ahom Anatolian_Hieroglyphs Hluw Arabic Arab Armenian Armn Avestan Avst Balinese Bali Bamum Bamu Bassa_Vah Bass Batak Batk Bengali Beng Bhaiksuki Bhks Bopomofo Bopo Brahmi Brah Braille Brai Buginese Bugi Buhid Buhd Canadian_Aboriginal Cans Carian Cari Caucasian_Albanian Aghb Chakma Cakm Cham Cham Cherokee Cher Common Zyyy Coptic Copt Qaac Cuneiform Xsux Cypriot Cprt Cyrillic Cyrl Deseret Dsrt Devanagari Deva Duployan Dupl Egyptian_Hieroglyphs Egyp Elbasan Elba Ethiopic Ethi Georgian Geor Glagolitic Glag Gothic Goth Grantha Gran Greek Grek Gujarati Gujr Gurmukhi Guru Han Hani Hangul Hang Hanunoo Hano Hatran Hatr Hebrew Hebr Hiragana Hira Imperial_Aramaic Armi Inherited Zinh Qaai Inscriptional_Pahlavi Phli Inscriptional_Parthian Prti Javanese Java Kaithi Kthi Kannada Knda Katakana Kana Kayah_Li Kali Kharoshthi Khar Khmer Khmr Khojki Khoj Khudawadi Sind Lao Laoo Latin Latn Lepcha Lepc Limbu Limb Linear_A Lina Linear_B Linb Lisu Lisu Lycian Lyci Lydian Lydi Mahajani Mahj Malayalam Mlym Mandaic Mand Manichaean Mani Marchen Marc Masaram_Gondi Gonm Meetei_Mayek Mtei Mende_Kikakui Mend Meroitic_Cursive Merc Meroitic_Hieroglyphs Mero Miao Plrd Modi Mongolian Mong Mro Mroo Multani Mult Myanmar Mymr Nabataean Nbat New_Tai_Lue Talu Newa Newa Nko Nkoo Nushu Nshu Ogham Ogam Ol_Chiki Olck Old_Hungarian Hung Old_Italic Ital Old_North_Arabian Narb Old_Permic Perm Old_Persian Xpeo Old_South_Arabian Sarb Old_Turkic Orkh Oriya Orya Osage Osge Osmanya Osma Pahawh_Hmong Hmng Palmyrene Palm Pau_Cin_Hau Pauc Phags_Pa Phag Phoenician Phnx Psalter_Pahlavi Phlp Rejang Rjng Runic Runr Samaritan Samr Saurashtra Saur Sharada Shrd Shavian Shaw Siddham Sidd SignWriting Sgnw Sinhala Sinh Sora_Sompeng Sora Soyombo Soyo Sundanese Sund Syloti_Nagri Sylo Syriac Syrc Tagalog Tglg Tagbanwa Tagb Tai_Le Tale Tai_Tham Lana Tai_Viet Tavt Takri Takr Tamil Taml Tangut Tang Telugu Telu Thaana Thaa Thai Thai Tibetan Tibt Tifinagh Tfng Tirhuta Tirh Ugaritic Ugar Vai Vaii Warang_Citi Wara Yi Yiii Zanabazar_Square Zanb"
const ecma10ScriptValues = ecma9ScriptValues + " Dogra Dogr Gunjala_Gondi Gong Hanifi_Rohingya Rohg Makasar Maka Medefaidrin Medf Old_Sogdian Sogo Sogdian Sogd"
const ecma11ScriptValues = ecma10ScriptValues + " Elymaic Elym Nandinagari Nand Nyiakeng_Puachue_Hmong Hmnp Wancho Wcho"
const ecma12ScriptValues = ecma11ScriptValues + " Chorasmian Chrs Diak Dives_Akuru Khitan_Small_Script Kits Yezi Yezidi"
const ecma13ScriptValues = ecma12ScriptValues + " Cypro_Minoan Cpmn Old_Uyghur Ougr Tangsa Tnsa Toto Vithkuqi Vith"
const unicodeScriptValues = {
9: ecma9ScriptValues,
10: ecma10ScriptValues,
11: ecma11ScriptValues,
12: ecma12ScriptValues
12: ecma12ScriptValues,
13: ecma13ScriptValues
}

const data = {}
function buildUnicodeData(ecmaVersion) {
let d = data[ecmaVersion] = {
const d = data[ecmaVersion] = {
binary: wordsRegexp(unicodeBinaryProperties[ecmaVersion] + " " + unicodeGeneralCategoryValues),
nonBinary: {
General_Category: wordsRegexp(unicodeGeneralCategoryValues),
Expand All @@ -46,9 +50,9 @@ function buildUnicodeData(ecmaVersion) {
d.nonBinary.sc = d.nonBinary.Script
d.nonBinary.scx = d.nonBinary.Script_Extensions
}
buildUnicodeData(9)
buildUnicodeData(10)
buildUnicodeData(11)
buildUnicodeData(12)

for (const ecmaVersion of [9, 10, 11, 12, 13]) {
buildUnicodeData(ecmaVersion)
}

export default data
20 changes: 0 additions & 20 deletions bin/test262.whitelist
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +0,0 @@
built-ins/RegExp/property-escapes/generated/Script_-_Cypro_Minoan.js (default)
built-ins/RegExp/property-escapes/generated/Script_-_Cypro_Minoan.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_-_Old_Uyghur.js (default)
built-ins/RegExp/property-escapes/generated/Script_-_Old_Uyghur.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_-_Tangsa.js (default)
built-ins/RegExp/property-escapes/generated/Script_-_Tangsa.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_-_Toto.js (default)
built-ins/RegExp/property-escapes/generated/Script_-_Toto.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_-_Vithkuqi.js (default)
built-ins/RegExp/property-escapes/generated/Script_-_Vithkuqi.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Cypro_Minoan.js (default)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Cypro_Minoan.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Old_Uyghur.js (default)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Old_Uyghur.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tangsa.js (default)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tangsa.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Toto.js (default)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Toto.js (strict mode)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Vithkuqi.js (default)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Vithkuqi.js (strict mode)