Feature request: be able to specify the retention time for blocks

[lzh-lab]

Is your feature request related to a problem? Please describe.
Now the compactor only compact the blocks, there is no retention.

Describe the solution you'd like

1. add a default retention config
2. add a api to update retention time range

[lzh-lab]

Is there any one work on this ?

[pracucci]

We had some preliminary discussions to add a per-tenant rentention support in the compactor, but we haven't started the work yet. If you plan to work on it, I would strongly recommend you to write a design document to share, so that we can discuss the design along with the community before jumping on coding it.

[lzh-lab]

I want to work on it.

Later I will write a preliminary design document with my understanding.

[lzh-lab]

I have written a initial design document #3638

[blind-oracle]

By the way, is there any retention policy at all for blocks storage?
Not per-tenant, but at least global?

[pracucci]

By the way, is there any retention policy at all for blocks storage?
Not per-tenant, but at least global?

The current approch for a global retention is to define a bucket policy to automatically delete blocks after X time.

[blind-oracle]

@pracucci Thanks.

But that would be a hard-delete, right?
The queriers would just lose those blocks right away without marking them as to-be-deleted as compactor does?
Wouldn't queries fail?

[pracucci]

But that would be a hard-delete, right?
The queriers would just lose those blocks right away without marking them as to-be-deleted as compactor does?
Wouldn't queries fail?

Yes, that would be an hard delete. However, Cortex has a config option to set how back data should be queried:

# Limit how long back data (series and metadata) can be queried, up until
# <lookback> duration ago. This limit is enforced in the query-frontend, querier
# and ruler. If the requested time range is outside the allowed range, the
# request will not fail but will be manipulated to only query data within the
# allowed time range. 0 to disable.
# CLI flag: -querier.max-query-lookback
[max_query_lookback: <duration> | default = 0s]

Let's say you wanna set a retention of 400 days. You can configure:

• -querier.max-query-lookback=9600h
• Bucket policy retention to 401 days, to have 1 day buffer and no failing query

[blind-oracle]

Thanks, got it! I was wondering how to do it properly :)

Yes, having per-tenant policies would be a very nice feature

[pracucci]

Yes, having per-tenant policies would be a very nice feature

There's a PR #3638 with a design doc proposal but looks a bit stuck. At Grafana Labs we've interest in it, so either way we'll build it pretty soon.