Skip to content

Commit

Permalink
Add Helm chart upgrade instructions (grafana#1565)
Browse files Browse the repository at this point in the history
* Start drafting Helm chart conversion

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Apply code review suggestions

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add helm upgrade instruction

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Continue Helm docs

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Complete draft of Helm chart migration

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Clarify that a string configuration is required

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use consistent string keys

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add additional notes

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Fix broken anchor

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add missing word

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Prefer "merge" over "incorporate"

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Note requirement to disable MinIO

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Advise use of namespace flag to ensure upgrade to the correct namespace

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Prefer instruction to run command

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use period to end full sentences

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Fix typo

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Clarify what is set

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Prefer "want" over "wish"

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Improve readability with a comma

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Clarify what the nameOverride setting is and what it affects

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add missing article

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
  • Loading branch information
3 people authored Mar 29, 2022
1 parent 56c7cfe commit 1fb9b04
Showing 1 changed file with 149 additions and 0 deletions.
149 changes: 149 additions & 0 deletions docs/sources/migration-guide/migrating-from-cortex.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,3 +181,152 @@ jb install github.com/grafana/mimir/operations/mimir-mixin@main
1. Apply the updated Jsonnet

To verify that the cluster is operating correctly, use the [monitoring mixin dashboards]({{< relref "../operators-guide/visualizing-metrics/dashboards/_index.md" >}}).

## Updating to Grafana Mimir using Helm

You can update to the Grafana Mimir Helm chart from the Cortex Helm chart.

### Before you begin

- Ensure that you are running the v1.4.0 release of the Cortex Helm chart.
- Ensure that you are running ingesters using a Kubernetes StatefulSet.

In the `values.yaml` file:

```
ingester:
statefulSet:
enabled: true
```

**To migrate to the Grafana Mimir Helm chart:**

1. Install the updated monitoring mixin.

a. Add the dashboards to Grafana. The dashboards replace your Cortex dashboards and continue to work for monitoring Cortex deployments.

> **Note:** Resource dashboards are now enabled by default and require additional metrics sources.
> To understand the required metrics sources, refer to [Additional resources metrics]({{< relref "../operators-guide/visualizing-metrics/requirements.md#additional-resources-metrics" >}}).
b. Install the recording and alerting rules into the ruler or a Prometheus server.

1. Run the following command to add the Grafana Helm chart repository:

```bash
helm repo add grafana https://grafana.github.io/helm-charts
```

1. Convert the Cortex configuration in your `values.yaml` file.

a. Extract the Cortex configuration and write the output to the `cortex.yaml` file.

```bash
yq -Y '.config' <VALUES YAML FILE> > cortex.yaml
```

b. Use `mimirtool` to update the configuration.

```bash
mimirtool config convert cortex.yaml
```

c. Place the updated configuration under the `mimir.config` key at the top level of your Helm values file.

> **Note:** The Grafana Mimir Helm chart expects the configuration as a string value.
> You can provide a literal block string with the `|` symbol.
In your Helm values file:

```yaml
mimir:
config: |
<CONFIGURATION FILE CONTENTS>
```
d. Merge the templated configuration from the `mimir-distributed` `values.yaml` file.
The Cortex Helm chart sets some additional configuration using flags.
The Grafana Mimir Helm chart sets that additional configuration in the configuration file.

- Set `frontend_worker.frontend_address` to `'{{ template "mimir.fullname" . }}-query-frontend-headless.{{ .Release.Namespace }}.svc:{{ include "mimir.serverGrpcListenPort" . }}'`.
- Set `ruler.alertmanager_url` to `'dnssrvnoa+http://_http-metrics._tcp.{{ template "mimir.fullname" . }}-alertmanager-headless.{{ .Release.Namespace }}.svc.cluster.local/alertmanager'`.
- If you want to use memberlist as the ring KV store, set `memberlist.join_members` to `['{{ include "mimir.fullname" . }}-gossip-ring']`.
- Append the caching configuration to `blocks_storage.bucket_store`.

A partial Helm values file with the changes incorporated looks similar to:

```yaml
mimir:
config: |
blocks_storage:
{{- if .Values.memcached.enabled }}
chunks_cache:
backend: "memcached"
memcached:
addresses: "dns+{{ .Release.Name }}-memcached.{{ .Release.Namespace }}.svc:11211"
max_item_size: {{ .Values.memcached.maxItemMemory }}
{{- end }}
{{- if index .Values "memcached-metadata" "enabled" }}
metadata_cache:
backend: "memcached"
memcached:
addresses: "dns+{{ .Release.Name }}-memcached-metadata.{{ .Release.Namespace }}.svc:11211"
max_item_size: {{ (index .Values "memcached-metadata").maxItemMemory }}
{{- end }}
{{- if index .Values "memcached-queries" "enabled" }}
index_cache:
backend: "memcached"
memcached:
addresses: "dns+{{ .Release.Name }}-memcached-queries.{{ .Release.Namespace }}.svc:11211"
max_item_size: {{ (index .Values "memcached-queries").maxItemMemory }}
{{- end }}
frontend_worker:
frontend_address: "{{ template "mimir.fullname" . }}-query-frontend-headless.{{ .Release.Namespace }}.svc:{{ include "mimir.serverGrpcListenPort" . }}"
memberlist:
join_members: ["{{ include "mimir.fullname" . }}-gossip-ring"]
ruler:
alertmanager_url: "dnssrvnoa+http://_http-metrics._tcp.{{ template "mimir.fullname" . }}-alertmanager-headless.{{ .Release.Namespace }}.svc.cluster.local/alertmanager"
```

e. Remove the original Cortex `$.config` member.

> **Note:** The `$` symbol refers to the top level of the values file.

f. Set the ingester `podManagementPolicy` to `"OrderedReady"`.
The Grafana Mimir chart prefers `"Parallel"` for faster scale up, but this field is immutable on an existing StatefulSet.

In your `values.yaml` file:

```yaml
ingester:
podManagementPolicy: "OrderedReady"
```

g. Set the `nameOverride` to `cortex`.
This configuration parameter ensures that resources have the same names as those created by the Cortex Helm chart and ensures Kubernetes performs a rolling upgrade of existing resources instead of creating new resources.

In your `values.yaml` file:

```yaml
nameOverride: "cortex"
```

h. Disable MinIO.
The Grafana Mimir Helm chart enables MinIO by default for convenience during first time installs.
If you are migrating from Cortex, use your existing object storage, and disable MinIO.

In your `values.yaml` file:

```yaml
minio:
enabled: false
```

1. Run the Helm upgrade with the Grafana Mimir chart.

> **Note:** The name of the release must match your Cortex Helm chart release.

```bash
helm upgrade <RELEASE> grafana/mimir [-n <NAMESPACE>]
```

To verify that the cluster is operating correctly, use the [monitoring mixin dashboards]({{< relref "../operators-guide/visualizing-metrics/dashboards/_index.md" >}}).

0 comments on commit 1fb9b04

Please sign in to comment.