Repository navigation
Avoid carrying empty overwrite manifests - #1154
Conversation
9cd26b2 to
3aabd27
Compare
tanmayrauth
left a comment
There was a problem hiding this comment.
LGTM.
Walked through the four (foundDeletedCount, len(notDeleted)) cases — only (0, 0) changes behavior, which is exactly the all-tombstone manifest from a prior snapshot, and that's right to drop.
iterManifestEntries(m, true) only filters EntryStatusDELETED (manifest.go:835), so an empty iteration is a reliable signal for "no live data here". The current overwrite's own tombstones are produced separately in overwriteFiles.deletedEntries, so this doesn't lose any deletion record. Older snapshots' manifest lists still reference the skipped manifests, so time-travel and expiration paths are unaffected.
Two small things in the test, both optional:
table/table_test.go:1322: the trailingcontinueis dead since it's the last statement in the loop body.table/table_test.go:1295:for i := range 5withi+1inside reads a bit oddly;for i := 1; i <= 5; i++would let you drop the+1.
|
@rraulinio let's get the suggestions that @tanmayrauth implemented first, then we'll merge |
|
@zeroshade Done. |
Signed-off-by: Raul <gfxsraul@gmail.com>
e8748f0 to
e11b9f3
Compare
Fixes #1153.
Overwrite commits read existing manifest entries with deleted entries filtered out. If no live entries remain, carrying that manifest forward keeps stale deletion history reachable in the current snapshot and causes repeated replace operations to grow the manifest list.
This change skips existing overwrite manifests that have no live entries after
discardDeleted=truefiltering. It does not rely on manifest-level count metadata, so zero-count inherited manifests that still contain live entries are still preserved.A regression test covers repeated
ReplaceDataFilesWithDataFilescalls and verifies the current snapshot keeps a bounded manifest list instead of accumulating old delete-only manifests.