-
Notifications
You must be signed in to change notification settings - Fork 13.6k
Closed
Labels
C-cleanupCategory: PRs that clean code up or issues documenting cleanup.Category: PRs that clean code up or issues documenting cleanup.
Description
Right now this is done while lexing:
rust/src/libsyntax/parse/lexer/mod.rs
Lines 423 to 435 in f861b6e
if old_ch_is_newline { | |
if self.save_new_lines_and_multibyte { | |
self.filemap.next_line(self.pos); | |
} | |
self.col = CharPos(0); | |
} else { | |
self.col = self.col + CharPos(1); | |
} | |
if new_ch_len > 1 { | |
if self.save_new_lines_and_multibyte { | |
self.filemap.record_multibyte_char(self.pos, new_ch_len); | |
} | |
} |
While that may seem like it avoids going twice over the file contents, lexing is slow, while newline and non-width-1 (which is also improperly computed by means of multi-byte characters, instead of the unicode-width
library) character finding can be optimized one way or another.
Cleaning this up would get rid of hacks such as the save_new_lines_and_multibyte
flag in the code linked above and having to reimplement this in anything else using CodeMap
outside of libsyntax.
Metadata
Metadata
Assignees
Labels
C-cleanupCategory: PRs that clean code up or issues documenting cleanup.Category: PRs that clean code up or issues documenting cleanup.