Proposal: introduce bucket index for the blocks storage #3554
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Marco Pracucci <marco@pracucci.com>
3 tasks
ranton256
approved these changes
Dec 8, 2020
|
|
||
| ## Out of scope | ||
|
|
||
| We believe the same technique described in this proposal could be applied to optimise the store-gateway too (we've built a [PoC](https://github.com/cortexproject/cortex/compare/pracucci:experiment-object-store-based-blocks-index)), however to keep the design doc easier to discuss we suggest to keep the store-gateway out of scope and address it in a follow-up design doc. |
There was a problem hiding this comment.
Is there any reason the store-gateway would need significant changes to how the index is loaded and used?
There was a problem hiding this comment.
Not significant changes, but the problem is that the store-gateway would have to preload all index-headers anyway, so the main objection would be "preloading all index-headers is still a pain, even with the bucket index in place". I have an idea to not preload all index-headers in the store-gateway, but I would prefer to discuss it in a separate design doc, to keep this doc lighter.
| There are few reasons why the compactor may be a good candidate to write the bucket index: | ||
|
|
||
| - The compactor already scans the bucket to have a consistent view of the blocks before running a compaction. Writing the bucket index in the compactor would require no additional bucket API calls. | ||
| - Queriers and store-gateways currently read only from the bucket (no writes). We believe that it may be a nice property to preserve. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
As previously shared in the CNCF Slack channel, I'm proposing to add a per-tenant bucket index to improve scalability of Cortex blocks storage when running a cluster with thousands of tenants. This PR is contains the design doc originally shared via google doc.
Which issue(s) this PR fixes:
N/A
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]