perf(immut): bulk build hash collections#3746
Open
mizchi wants to merge 1 commit into
Open
Conversation
mizchi
marked this pull request as ready for review
June 29, 2026 11:17
mizchi
marked this pull request as draft
June 29, 2026 11:22
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves construction performance for immutable HAMT-backed HashMap and HashSet by introducing a bulk-build path for from_array / from_iter, avoiding repeated persistent add work and reducing allocations during branch construction.
Changes:
- Added a bulk HAMT builder that partitions entries by path segment and builds branch nodes from contiguous ranges, with a small-input fast path (≤ 64) that keeps the existing
addloop. - Introduced
@immut/internal/sparse_array.from_sorted_fixed_arrayto buildSparseArraychildren in a single allocation during bulk construction. - Added build benchmarks and extended tests to ensure duplicate-handling semantics remain unchanged.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| immut/internal/sparse_array/sparse_array.mbt | Adds from_sorted_fixed_array to support single-allocation sparse-array construction for bulk HAMT builds. |
| immut/internal/sparse_array/sparse_array_test.mbt | Adds coverage for SparseArray::from_sorted_fixed_array. |
| immut/internal/sparse_array/pkg.generated.mbti | Exposes the new sparse-array constructor in the generated interface. |
| immut/hashset/moon.pkg | Imports bench for test/bench harness integration. |
| immut/hashset/hashset_build_bench_test.mbt | Adds benchmarks for HashSet::from_array / from_iter bulk construction performance. |
| immut/hashset/HAMT.mbt | Implements bulk-build construction for HashSet::from_array / from_iter using segmented partitioning and sparse-array bulk child creation. |
| immut/hashset/HAMT_test.mbt | Adds tests to verify bulk-build matches prior add-based behavior, including duplicate representative semantics. |
| immut/hashmap/moon.pkg | Imports bench for test/bench harness integration. |
| immut/hashmap/hashmap_build_bench_test.mbt | Adds benchmarks for HashMap::from_array / from_iter bulk construction performance. |
| immut/hashmap/HAMT.mbt | Implements bulk-build construction for HashMap::from_array / from_iter, preserving first/last duplicate semantics as before. |
| immut/hashmap/HAMT_test.mbt | Adds tests to verify bulk-build matches prior add-based behavior, including duplicate value semantics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mizchi
marked this pull request as ready for review
June 29, 2026 11:23
mizchi
force-pushed
the
perf/hamt-build
branch
4 times, most recently
from
July 5, 2026 15:47
af17be3 to
fb44460
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
HashMap::from_array/from_iterandHashSet::from_array/from_iter.addwork for larger inputs.SparseArray::addpath during bulk construction by adding@immut/internal/sparse_array.from_sorted_fixed_arrayand building branch children in one allocation.BuildEntryarrays forfrom_array, and use iteratorsize_hintas capacity forfrom_iter.addloop for small inputs (<= 64) to avoid bulk-builder overhead.HashMap::from_arrayremains first value wins.HashMap::from_iterremains last value wins.HashSet::from_array/from_iterkeep their existing duplicate representative behavior.benchclock helpers and JSBigIntAPIs that were surfaced bymoon check.Profile findings
moon test --profilerequiresxcrun xctrace; on this machine I ran it with:The temporary profile drivers repeatedly built a 10k-element collection and were not committed.
Before the
SparseArraychange, the profile showedSparseArray::addas a real hotspot:SparseArray::addwas 17.5% inclusive, mostlymoonbit_make_ref_array_with_blit/moonbit_unsafe_ref_array_blit.SparseArray::addwas 18.1% inclusive with the same copy/blit cost.After constructing sparse arrays from sorted child indices,
SparseArray::adddisappeared from the top profile entries. Samples for the same profile driver dropped:The remaining top cost is now
build_*_node_rangeplus allocation/drop overhead while partitioning each HAMT level.Benchmarks
moon bench -p immut/hashmap --target nativeHashMap::from_array ordered n=10000HashMap::from_array random n=10000HashMap::from_iter random n=10000moon bench -p immut/hashset --target nativeHashSet::from_array ordered n=10000HashSet::from_array random n=10000HashSet::from_iter random n=10000Validation
moon fmtmoon infomoon check -p bench --target allmoon check -p bigint --target allmoon check -p immut/internal/sparse_array --target allmoon check -p immut/hashmap --target allmoon check -p immut/hashset --target allmoon test -p bench --target allmoon test -p bigint --target allmoon test -p immut/internal/sparse_array --target allmoon test -p immut/hashmap --target allmoon test -p immut/hashset --target allmoon bench -p immut/hashmap --target nativemoon bench -p immut/hashset --target nativegit diff --checkThe missing-doc warnings seen earlier are fixed.