Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build strings without repeated re-allocation and copies #37

Open
overlookmotel opened this issue Jun 6, 2024 · 0 comments
Open

Build strings without repeated re-allocation and copies #37

overlookmotel opened this issue Jun 6, 2024 · 0 comments

Comments

@overlookmotel
Copy link

overlookmotel commented Jun 6, 2024

Same problem as #35, but difference is that once strings/Atoms are generated, they're immutable. So there's a simpler solution:

Assume we're going to create our own arena allocator.

Fill the allocator from both ends:

  • Fill from end for AST nodes (bumping downwards, same as bumpalo does).
  • Fill from start for strings (bumping upwards).

Same sort of idea as how heap grows upwards, and stack grows downwards.

We only build 1 string at a time, so can always grow strings in place while building them. This avoids memory fragmentation, and unused gaps in arena.

Side benefit: All string data in one place means AST transfer can convert all strings from UTF8 to UTF16 in one go (still complicated due to needing UTF16 indexes to slice that string, but it's a step in right direction).

@overlookmotel overlookmotel changed the title BuildStrings without repeated re-allocation and copies Build strings without repeated re-allocation and copies Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant