Skip to content

Automatic blob garbage collection #157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 98 commits into from
Oct 12, 2025
Merged

Automatic blob garbage collection #157

merged 98 commits into from
Oct 12, 2025

Conversation

marvin-j97
Copy link
Contributor

@marvin-j97 marvin-j97 commented Sep 28, 2025 •
edited
Loading

  • Add blob fragmentation map to Version (+ ser/de)
  • Add optional callback to CompactionStream to listen for expired KVs
  • Install callback to listen for expired blobs if tree is key-value separated
  • Store referenced blob file IDs in table file on flush and compaction
  • Remove all superseded old vlog + GC code
  • Add a bunch of tests for blob tree

@marvin-j97 marvin-j97 added this to the 3.0.0 milestone Sep 28, 2025
@marvin-j97 marvin-j97 self-assigned this Sep 28, 2025
@marvin-j97 marvin-j97 changed the base branch from main to 3.0.0 September 28, 2025 15:26
@marvin-j97 marvin-j97 marked this pull request as ready for review October 12, 2025 14:19
@marvin-j97 marvin-j97 merged commit 40032a4 into 3.0.0 Oct 12, 2025
11 checks passed
@marvin-j97 marvin-j97 deleted the auto-gc branch October 12, 2025 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@marvin-j97 marvin-j97

Labels

api enhancement New feature or request epic performance refactor test type:compaction type:table type:vlog

Projects

None yet

Milestone

3.0.0

Development

Successfully merging this pull request may close these issues.

1 participant

@marvin-j97