Skip searching through lsm tree for a target level when files overlap #12284
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jowlyzhang
force-pushed
the
files_overlap_skip_search_lsm
branch
from
548e176
to
4037fca
Compare
ajkr
reviewed
Jan 24, 2024
|
@jowlyzhang has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@jowlyzhang merged this pull request in 928aca8. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Feb 3, 2024
Summary: Updates some documentations and invariant assertions after #12257 and #12284. Also refactored some duplicate code and improved some error message and preconditions for errors. Pull Request resolved: #12305 Test Plan: Existing unit tests Reviewed By: hx235 Differential Revision: D53371325 Pulled By: jowlyzhang fbshipit-source-id: fb0edcb3a3602cdf0a292ef437cfdfe897fc6c99
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While ingesting multiple external files with key range overlap, current flow go through the lsm tree to do a search for a target level and later discard that result by defaulting back to L0. This PR improves this by just skip the search altogether.
The other change is to remove default to L0 for the combination of universal compaction + force global sequence number, which was initially added to meet a pre #7421 invariant.
Test Plan:
Added unit test:
./external_sst_file_test --gtest_filter="IngestFileWithGlobalSeqnoAssignedUniversal"