Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Unicode support for identifier names #1003

Merged
merged 25 commits into from
Jan 10, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add README.md and link to CONTRIBUTING.md
  • Loading branch information
jevancc committed Jan 10, 2021
commit 4e1ecb2b7780710a153009001a8eaa76ee0401fb
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ See [Debugging](./docs/debugging.md).
If you want to develop on the web assembly side you can run `yarn serve` and then go
to <http://localhost:8080>.

### boa-unicode
Boa uses the library `boa-unicode` to query Unicode character properties and classes in lexer and parser. See [boa_unicode/README.md](./boa_unicode/README.md) for development and more information.

### Setup

#### VSCode Plugins
Expand Down
21 changes: 21 additions & 0 deletions boa_unicode/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# boa-unicode

`boa-unicode` defines the trait to provide methods for querying properties or classes for Unicode identifiers. These properties are used to determine if a code point (char) is valid for being the start/part of an identifier in lexer and parser.

Current version: Unicode 13.0.0

## Development
The Unicode character tables used to query properties are generated by `build_tables.js`. This script depends on [Node.js](https://nodejs.org/en/) and [rustfmt](https://github.com/rust-lang/rustfmt). You can run the script with:
```
$ node build_tables.js
```
or with [npm](https://www.npmjs.com/):
```
$ npm run build-tables
```

The configurations are defined as constants in the script. Please check the comments in `build_tables.js` for more information.

## More Info
- [Unicode® Standard Annex #31 - UNICODE IDENTIFIER AND PATTERN SYNTAX](https://unicode.org/reports/tr31/)
- [Unicode® Standard Annex #44 - UNICODE CHARACTER DATABASE](https://unicode.org/reports/tr44/)
2 changes: 1 addition & 1 deletion boa_unicode/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! This crate implements the extension to query if a char belongs to a particular unicode identifier property.
//! This library implements the extension to query if a char belongs to a particular unicode identifier property.
//! Version: Unicode 13.0.0
//!
//! More information:
Expand Down