Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compactor removes blocks within retention period #7064

Open
Kot-o-pes opened this issue Jan 16, 2024 · 0 comments
Open

Compactor removes blocks within retention period #7064

Kot-o-pes opened this issue Jan 16, 2024 · 0 comments

Comments

@Kot-o-pes
Copy link

Kot-o-pes commented Jan 16, 2024

Thanos, Prometheus and Golang version used:

thanos, version 0.32.5 (branch: HEAD, revision: 750e8a94eed5226cd4562117295d540a968c163c)
  build user:       root@053ebc7b5322
  build date:       20231019-04:13:41
  go version:       go1.21.3
  platform:         linux/amd64
  tags:             netgo

Object Storage Provider: Minio

What happened: Compactor runs with flags

--retention.resolution-raw=30d --retention.resolution-5m=180d --retention.resolution-1h=1y --log.format=json --wait --deduplication.replica-label=receive_replica --deduplication.replica-label=replica --deduplication.replica-label=prometheus_replica  --compact.concurrency=20 --block-files-concurrency=20 --downsample.concurrency=20 --compact.enable-vertical-compaction --deduplication.func=penalty 

considering 5m retention i do not expect blocks within this range to be deleted, but it looks like blocks that fall behind 60d are being deleted though 60d retention is not mentioned anywhere
first block from thanos tools bucket inspect
| 01HFYCTQDYAKH1PBSSRV3BQQFR | 2023-11-15T03:00:00+03:00 | 2023-11-23T03:00:00+03:00 | 191h59m59.947s | 48h0m0.053s | 151,016 | 287,782,024 | 2,145,069 | 4 | false | cluster=k8s,environment=prod,prometheus=monitoring/prom-operator-prometheus | 5m0s | compactor |
which is exactly 60d after today (2024-01-15)

What you expected to happen: retention does not delete blocks within set range

How to reproduce it (as minimally and precisely as possible):

Full logs to relevant components:

Anything else we need to know:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants