turbo-persistence: add key-value tombstones for MultiValue families - #96929
Draft
lukesandberg wants to merge 1 commit into
Draft
turbo-persistence: add key-value tombstones for MultiValue families#96929lukesandberg wants to merge 1 commit into
lukesandberg wants to merge 1 commit into
Conversation
Contributor
Tests PassedCommit: 187f9dd |
Contributor
Stats from current PR✅ No significant changes detected📊 All Metrics📖 Metrics GlossaryDev Server Metrics:
Build Metrics:
Change Thresholds:
⚡ Dev Server
📦 Dev Server (Webpack) (Legacy)📦 Dev Server (Webpack)
⚡ Production Builds
📦 Production Builds (Webpack) (Legacy)📦 Production Builds (Webpack)
📦 Bundle SizesBundle Sizes⚡ TurbopackClient Main Bundles
Server Middleware
Build DetailsBuild Manifests
Build Cache
📦 WebpackClient Main Bundles
Polyfills
Pages
Server Edge SSR
Middleware
Build DetailsBuild Manifests
Build Cache
🔄 Shared (bundler-independent)Runtimes
📎 Tarball URLCommit: 187f9dd |
lukesandberg
force-pushed
the
lukesandberg/gc-015-valued-tombstones
branch
2 times, most recently
from
August 7, 2026 19:03
d1c86a5 to
846b01e
Compare
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| .map(|&i| ssts_with_ranges[i].seq) | ||
| .min() | ||
| .unwrap_or(0); | ||
| let older_filters = ssts_with_ranges |
Contributor
There was a problem hiding this comment.
During compaction, tombstone_is_dead uses a seq < min(job seq) cut that can drop a still-needed key/key-value tombstone while an intervening overlapping SST outside the merge job still holds the deleted value, resurrecting it on lookup.
Removing one value from a MultiValue key currently requires a read-modify-write: `delete` tombstones the whole hash bucket, so the caller must first read the bucket back and re-insert every surviving value. At GC scale that is one synchronous bucket read per deleted key. Add `WriteBatch::delete_value`, which writes a tombstone carrying the value being deleted. The delete is optimistic: nothing is read at write time, and the tombstone is applied lazily by reads and by compaction. Deleting N pairs costs N buffered writes instead of N read-modify-write cycles. The two tombstone kinds now sit at opposite ends of a key group. Key-value tombstones sort first, so a reader collects them on entering the group and filters the values that follow in one forward pass. Key tombstones keep sorting last, because they mean "everything older than me is gone" including same-SST entries: a batch doing `put(A); delete; put(B)` must still keep A and B while shadowing older SSTs. Knowing when a tombstone can be dropped was already an unsolved problem for key tombstones -- today they are simply kept forever -- and key-value tombstones inherit it. Rather than require a full-family rewrite to prove no older layer holds the key, compaction now probes the AMQF filters of the SSTs older than the job. The filter's error is one-sided (`false` means definitely absent), so a false positive only retains a tombstone for another round and the resurrection direction is unreachable. Filters are already resident and range-pruned, so this is in-memory work on the entries that need it. This reclaims tombstones incrementally, with no full-compaction cycle, no new metadata, and no unchecked caller obligation. Only SSTs older than the whole job are consulted: within the job, MergeIter already yields newest-first and drops the older values a tombstone shadows. It applies to both kinds, so key tombstones become reclaimable too. The on-disk format is unchanged apart from the new key-value tombstone entry type, which takes a previously unused tag, so existing databases stay readable.
lukesandberg
force-pushed
the
lukesandberg/gc-015-valued-tombstones
branch
from
August 7, 2026 20:28
846b01e to
187f9dd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.