Remove bloom compactor component

[chaudum]

Background

Some time ago, a two new components for building bloom filters were introduced: the bloom planner and the bloom builder.

These new components fulfil the same functionality as the the compactor, however, they come with certain advantages:

• The consistent hash ring for the compactors is removed. The ring is hard to operate and configure and causes problems with the ownership of fingerprint ranges when instances are added and removed regularly (e.g. due to OOMing)
• The planner/builder approach can distribute load better, because builders get new tasks from the queue whenever their previous task is completed. This reduces idle time of instances.
• Only the planner is "stateful", since it keeps the queue. Builders are stateless and can be scaled horizontally depending on the needs, without changing the fingerprint ranges of the tasks (which was the case when running compactors). This can enable huge cost savings, because planners can be run on spot instances.

List of PRs

• feat: Boilerplate for new bloom build planner and worker components. #12989
• refactor(bloom planner): Compute gaps and build tasks from metas and TSDBs  #12994
• refactor(blooms): Add queue to bloom planner and enqueue tasks #13005
• refactor(blooms): Add RPC service for bloom-planner #13015
• refactor(blooms): Builder retrieves tasks from planner #13046
• refactor(blooms): Implement retry mechanisms in planner #13064
• refactor(blooms): Build new metas and blocks #13074
• refactor(blooms): Add metrics for per-tenant tasks progress to planner #13078
• refactor(blooms): Forward created metas from builder to planner #13133
• refactor(blooms): Limit task retries in bloom planner #13139
• refactor(blooms): Delete outdated metas #13153
• refactor(blooms): Implement retry in builder #13306
• refactor(blooms): Better task naming #13361
• fix(blooms): Delete outdated metas during planning #13363
• refactor(blooms): Apply retention in planner #13484
• fix(blooms): Remove backoff from notify planner #13506
• fix(blooms): Ship chunkrefs in task payload #13677

Change request

⚠️ Note
Bloom filters are an experimental feature and therefore subject to rapid changes.

Because of these new components, the bloom compactor should not be considered any more and existing deployments should be migrated as soon as possible.

TODOs

• Update documentation to reference bloom planner and builders
• Integrate bloom building into SSD deployment
• Remove all bloom compactor related code
• Update Helm chart (distributed) to use the new components