Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lazy load chunks #435

Merged
merged 5 commits into from
Apr 25, 2019
Merged

Lazy load chunks #435

merged 5 commits into from
Apr 25, 2019

Conversation

gouthamve
Copy link
Member

Wanted to test this first before I pull in all my lazy loading logic.

@gouthamve gouthamve force-pushed the lazy-iteration branch 3 times, most recently from 72ca3f1 to 2bed34f Compare April 3, 2019 09:08
@gouthamve
Copy link
Member Author

This is good for review, current issues:

  1. The chunks are loaded sequentially and one by one and only when accessed. This means when the heap iterator calls Next(), the chunks will loaded.
  2. We do need to load the chunks for the number of series that exist. That means we will need to load gigabytes into memory.
  3. Need to implement filterChunksByMatchers again. But first would like a review of this before I do that.

@gouthamve gouthamve changed the title WIP: Lazy load chunks Lazy load chunks Apr 19, 2019
@gouthamve
Copy link
Member Author

@tomwilkie This is good for review now!

cyriltovena
cyriltovena reviewed Apr 23, 2019
View reviewed changes
cyriltovena
cyriltovena reviewed Apr 23, 2019
View reviewed changes
Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, got some questions and requests though.

@gouthamve gouthamve force-pushed the lazy-iteration branch 2 times, most recently from 89bd1e6 to 22fd67e Compare April 24, 2019 15:52
@gouthamve
Lazy load chunks for queries.
d6a5341 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@gouthamve
Add context to the lazy chunk calls
7b92c94 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@gouthamve
Make sure labels are plumbed through
cf9e75c 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@gouthamve
Pre-load non-overlapping chunk iterators
b12c8ce 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@gouthamve
Review feedback
1c99f51 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@gouthamve gouthamve merged commit 27ca58e into grafana:master Apr 25, 2019
@gouthamve gouthamve deleted the lazy-iteration branch April 25, 2019 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cyriltovena cyriltovena cyriltovena approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gouthamve @cyriltovena