Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs(thanos): add more context and quick start guide #477

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
+964 −25
Open

docs(thanos): add more context and quick start guide #477

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
cbc630b
docs(thanos): add more context and quick start guide
richardtief Nov 4, 2024
cbb3280
docs(thanos): img formatting
richardtief Nov 4, 2024
d67df12
docs(thanos): Apply suggestions from review
richardtief Nov 4, 2024
291de9a
chore(licenserc): ignore thanos dir
richardtief Nov 4, 2024
920a0fc
docs(thanos): remove intro paragraph
richardtief Nov 4, 2024
b4a58c1
docs(thanos): refresh arch drawings
richardtief Nov 7, 2024
7a6a0bb
docs(thanos): update arch drawing
richardtief Nov 8, 2024
393a671
docs(thanos): add component description and interplay
richardtief Nov 8, 2024
ed451eb
docs(thanos): move arch drawing to the top
richardtief Nov 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 60 additions & 22 deletions thanos/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,63 @@
title: Thanos
---

# Information
Learn more about the **Thanos** Plugin. Use it to enable extended metrics retention and querying across Prometheus servers and Greenhouse clusters.

The main terminologies used in this document can be found in [core-concepts](https://cloudoperators.github.io/greenhouse/docs/getting-started/core-concepts).

## Overview

Observability is often required for operation and automation of service offerings. To get the insights provided by an application and the container runtime environment, you need telemetry data in the form of _metrics_ or _logs_ sent to backends such as _Prometheus_ or _OpenSearch_. With the **Thanos** plugin you can scale the _metric_ part of the observability stack horizontally by hooking into the Prometheus provided by the **kube-monitoring** plugin and optionally save your data in the object storage.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Observability is often required for operation and automation of service offerings. To get the insights provided by an application and the container runtime environment, you need telemetry data in the form of _metrics_ or _logs_ sent to backends such as _Prometheus_ or _OpenSearch_. With the **Thanos** plugin you can scale the _metric_ part of the observability stack horizontally by hooking into the Prometheus provided by the **kube-monitoring** plugin and optionally save your data in the object storage.
Observability is required for monitoring the lifecycle of service offerings. To get the insights provided by an application and the container runtime environment, you need telemetry data in the form of _metrics_ or _logs_ which are made accessible through backends such as _Prometheus_ or _OpenSearch_.
With the **Thanos** plugin you can enhance the _metric_ part of the observability stack to scale horizontally and leverage object storage for a long retention setup.

What explicitly do you mean with container runtime environment?

To me this is/was somehow hard to read and grasp. I rewrote it, not sure if this is understandable - or even needed. The following section (In general, Thanos is...)is more to the point, while this is too generic. I would wipe it entirely. Similar wording could go in an overall metrics architecture document.

Furthermore, I don't see why logs w/ need to be mentioned here, this also would rather be part of an overarching document.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubernetes. 🤣

What explicitly do you mean with container runtime environment?

You're right, I admit. We don't have space for such a general document. That's why it's included in every plugin at the moment. We can also remove it and I will find a place for it.

To me this is/was somehow hard to read and grasp. I rewrote it, not sure if this is understandable - or even needed. The following section (In general, Thanos is...)is more to the point, while this is too generic. I would wipe it entirely. Similar wording could go in an overall metrics architecture document.
Furthermore, I don't see why logs w/ need to be mentioned here, this also would rather be part of an overarching document.


In general, Thanos is a set of components that can be used to extend the storage and retrieval of metrics in Prometheus. It allows you to store metrics in a remote object store and query them across multiple Prometheus servers and Greenhouse clusters. This Plugin is intended to provide a set of pre-configured Thanos components that enable a proven composition. At the core, a set of Thanos components is installed that adds long-term storage capability to a single **kube-monitoring** Plugin and makes both current and historical data available again via one Thanos Query component.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am now asking myself, if this is sort of a repetition of the very first sentence

Learn more about the Thanos Plugin. Use it to enable extended metrics retention and querying across Prometheus servers and Greenhouse clusters.

This part could easily go there:

This Plugin is intended to provide a set of pre-configured Thanos components that enable a proven composition. At the core, a set of Thanos components is installed that adds long-term storage capability to a single kube-monitoring Plugin and makes both current and historical data available again via one Thanos Query component.

We could however add some prose on how the components are acting together here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. We can improve.

I am now asking myself, if this is sort of a repetition of the very first sentence

Good idea. I'm open for a contribution from your side.

We could however add some prose on how the components are acting together here?


This plugin deploys the following Thanos components:

* Query
* Compact
* Store
<!--* Query Frontend-->
<!--* (Ruler)-->
* [Thanos Query](https://thanos.io/tip/components/query.md/)
* [Thanos Store](https://thanos.io/tip/components/store.md/)
* [Thanos Compactor](https://thanos.io/tip/components/compact.md/)

Planned components:
* [Thanos Ruler](https://thanos.io/tip/components/rule.md/)
* [Thanos Query Frontend](https://thanos.io/tip/components/query.md/)

This Plugin does **not** deploy the following components:
* [Thanos Sidecar](https://thanos.io/tip/components/sidecar.md/)
This component is installed in the [kube-monitoring](https://github.com/cloudoperators/greenhouse-extensions/tree/main/kube-monitoring) plugin.

## Note

It is not meant to be a comprehensive package that covers all scenarios. If you are an expert, feel free to configure the Plugin according to your needs.

Contribution is highly appreciated. If you discover bugs or want to add functionality to the plugin, then pull requests are always welcome.

## Quick start

Requirements (detailed steps below):
* ready to use credentials for a [compatible object store](https://thanos.io/tip/thanos/storage.md/)
* [thanos-sidecar enabled in Prometheus](#kube-monitoring-plugin-enablement) (usually with [Prometheus Operator](https://prometheus-operator.dev/docs/api-reference/api/#monitoring.coreos.com/v1.ThanosSpec)).
This guide provides a quick and straightforward way to use **Thanos** as a Greenhouse Plugin on your Kubernetes cluster. The guide is meant to build the following setup:

# Owner
![Thanos Architecture](./img/Thanos-setup-example.png)
richardtief marked this conversation as resolved.
Show resolved Hide resolved

1. Tommy Sauer (@viennaa)
2. Richard Tief (@richardtief)
3. Martin Vossen (@artherd42)
**Prerequisites**

# Configuration
- A running and Greenhouse-onboarded Kubernetes cluster
richardtief marked this conversation as resolved.
Show resolved Hide resolved
- Ready to use credentials for a [compatible object store](https://thanos.io/tip/thanos/storage.md/)
- **kube-monitoring** plugin installed. Thanos Sidecar on the Prometheus must be [enabled](#kube-monitoring-plugin-enablement) by providing the required object store credentials.

## Object Store preparation
**Step 1:**

Create a Kubernetes Secret with your object store credentials following the [Object Store preparation](#object-store-preparation) section.

**Step 2:**

Enable the Thanos Sidecar on the Prometheus in the **kube-monitoring** plugin by providing the required object store credentials. Follow the [kube-monitoring plugin enablement](#kube-monitoring-plugin-enablement) section.

**Step 3:**

Create a Thanos Query Plugin by following the [Thanos Query](#thanos-query) section.

## Configuration

### Object Store preparation

To run Thanos, you need object storage credentials. Get the credentials of your provider and add them to a Kubernetes Secret. The [Thanos documentation](https://thanos.io/tip/thanos/storage.md/) provides a great overview on the different supported store types.

Expand All @@ -47,8 +81,10 @@ If you've got everything in a file, deploy it in your remote cluster in the name
kubectl create secret generic $THANOS_PLUGIN_NAME-metrics-objectstore --from-file=thanos.yaml=/path/to/your/file
```

### kube-monitoring plugin enablement

(usually with [Prometheus Operator](https://prometheus-operator.dev/docs/api-reference/api/#monitoring.coreos.com/v1.ThanosSpec)).
richardtief marked this conversation as resolved.
Show resolved Hide resolved

## kube-monitoring plugin enablement

Prometheus in kube-monitoring needs to be altered to have a sidecar and ship metrics to the new object store too. You have to provide the Secret you've just created to the (most likely already existing) kube-monitoring plugin. Add this:

Expand All @@ -61,7 +97,7 @@ spec:
value: $THANOS_PLUGIN_NAME-metrics-objectstore
```

## Thanos Query
### Thanos Query

This is the real deal now: Define your Thanos Query by creating a plugin.

Expand All @@ -81,8 +117,8 @@ spec:
releaseNamespace: kube-monitoring
```

## [OPTIONAL] Handling your Prometheus and Thanos Stores.
### Default Prometheus and Thanos Endpoint
### [OPTIONAL] Handling your Prometheus and Thanos Stores.
#### Default Prometheus and Thanos Endpoint

Thanos Query is automatically adding the Prometheus and Thanos endpoints. If you just have a single Prometheus with Thanos enabled this will work out of the box. Details in the next two chapters. See [Standalone Query](#standalone-query) for your own configuration.

Expand All @@ -100,7 +136,9 @@ If you just have one occurence of this Thanos plugin dpeloyed, the default optio

### Standalone Query

In case you want to have a Query to run with multiple Stores, you can set it to standalone and add your own store list. Set up your plugin like this:
![Standalone Query](./img/Thanos-standalone-query.png)

In case you want to achive a Setup like the above and have a Thanos Query to run with multiple Stores, you can set it to `standalone` and add your own store list. Setup your Plugin like this:
richardtief marked this conversation as resolved.
Show resolved Hide resolved

```yaml
spec:
Expand Down Expand Up @@ -134,9 +172,9 @@ This would enable you to either:
- thanos-kube-2:10901
```

# Operations
## Operations

## Thanos Compactor
### Thanos Compactor

If you deploy the plugin with the default values, Thanos compactor will be shipped too and use the same secret (`$THANOS_PLUGIN_NAME-metrics-objectstore`) to retrieve, compact and push back timeseries.

Expand Down
Loading
Loading