Skip to content

Conversation

@N-o-Z
Copy link
Member

@N-o-Z N-o-Z commented Jan 28, 2026

Proposal for branch read optimization

@N-o-Z N-o-Z requested review from a team and ozkatz January 28, 2026 02:06
@N-o-Z N-o-Z self-assigned this Jan 28, 2026
@N-o-Z N-o-Z added proposal exclude-changelog PR description should not be included in next release changelog minor-change Used for PRs that don't require issue attached labels Jan 28, 2026
Comment on lines 197 to 200
if !branch.Dirty {
branch.Dirty = true
// This triggers a branch record update
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too late to update it now. There's already an entry in staging. If another read request comes, it should be able to see it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely right - we must first set dirty bit before any write

@N-o-Z N-o-Z requested a review from itaiad200 January 29, 2026 02:04
@ozkatz
Copy link
Collaborator

ozkatz commented Feb 2, 2026

Out of curiosity - do we know how common this case is? (i.e. % of reads that go to "clean" branches?) Might help us understand if the added complexity is worth it.

Perhaps a more granular approach could be beneficial: we can go further than a single boolean and maintain a small bitmap: one bit per range - essentially lakeFS' take on dirty pages.
Instead of reading the bool, read the bitmap; and if the requested range(s) are 0, you're good to read just the underlying committed data. if it's 1, check staging. on write, mark 1 for any range affected by the write.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

exclude-changelog PR description should not be included in next release changelog minor-change Used for PRs that don't require issue attached proposal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants