Move char/unified conversion into precompiled Util module #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Madness? Although... even after spending a day working on this, I'm currently leaning towards the tradeoff in code complexity not being worth it until someone actually hits this as the performance bottleneck, and leaving this unmerged. My guess is even in a tight loop the I/O might be more of a bottleneck at this point? Comments appreciated. |
madness? THIS IS SPARTA
don’t actually need to define variants first for `char_to_unified` because unlike `Scanner.bscan/1`, this is matching against an entire binary rather than just head, so no need to worry about similar out of order issues.
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Move the
.char_to_unifiedand.unified_to_charfundamental conversions into their own module, and then precompile the results for all known Emoji character values. This makes these conversions effectively zero-cost for all known Emoji values. If nothing is matched, falls back to the actual conversion functions.This adds a fair amount of code complexity, and because to precompile stuff you need any function it depends on to be in a different module, hence the nested Util modules here.
Speed comparison can be seen here, check out the Exmoji PR#1 column:
https://docs.google.com/spreadsheets/d/1T08I6dlyFNqqdtvQykNt43tdT85lJy4US2SqErcpZ7g/edit?usp=sharing
Most interesting benefit I see in the speed increases is the knock-on effects in making
.scan/1even faster, so the amount of text that could be scanned by a single node is even higher. That said, we were already pretty darn fast, so the code complexity (especially the potential to introduce ordering gotchas in compilation) might not be worth it here.