Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Create top-level Setup section #9974

Merged
merged 8 commits into from
Jul 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ Fixes #<issue number>
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](https://github.com/grafana/loki/commit/d10549e3ece02120974929894ee333d07755d213)
8 changes: 4 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -1371,7 +1371,7 @@ There are many other noteworthy additions and fixes, too many to list, but we sh

Thanks to everyone for another exciting Loki release!!

Please read the [Upgrade Guide](https://github.com/grafana/loki/blob/master/docs/sources/upgrading/_index.md#220) before upgrading for a smooth experience.
Please read the [Upgrade Guide](https://github.com/grafana/loki/blob/master/docs/sources/setup/upgrade/_index.md#220) before upgrading for a smooth experience.

TL;DR Loki 2.2 changes the internal chunk format which limits what versions you can downgrade to, a bug in how many queries were allowed to be scheduled per tenant was fixed which might affect your `max_query_parallelism` and `max_outstanding_per_tenant` settings, and we fixed a mistake related `scrape_configs` which do not have a `pipeline_stages` defined. If you have any Promtail `scrape_configs` which do not specify `pipeline_stages` you should go read the upgrade notes!

Expand Down Expand Up @@ -1831,11 +1831,11 @@ Thanks again for the many incredible contributions and improvements from the won

**Please Note** There are several changes in this release which require your attention!

* Anyone using a docker image please go read the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/upgrading/_index.md#200)!! There is one important consideration around a potentially breaking schema change depending on your configuration.
* MAJOR changes have been made to the boltdb-shipper index, breaking changes are not expected but extra precautions are highly recommended, more details in the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/upgrading/_index.md#200).
* Anyone using a docker image please go read the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/setup/upgrade/_index.md#200)!! There is one important consideration around a potentially breaking schema change depending on your configuration.
* MAJOR changes have been made to the boltdb-shipper index, breaking changes are not expected but extra precautions are highly recommended, more details in the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/setup/upgrade/_index.md#200).
* The long deprecated `entry_parser` config in Promtail has been removed, use [pipeline_stages](https://grafana.com/docs/loki/latest/clients/promtail/configuration/#pipeline_stages) instead.

Check the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/upgrading/_index.md#200) for detailed information on all these changes.
Check the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/setup/upgrade/_index.md#200) for detailed information on all these changes.

### 2.0!!!!

Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ LIDs must be created as a pull request using [this template](docs/sources/commun
* It fixes an issue present in a previous release
* It causes a change in operation that would be useful for an operator of Loki to know
* You can skip this step for documentation changes, build related changes and simple bug fixes or enhancements. Rationale being we are attempting to curate the CHANGELOG entries with the most relevant and important changes that end users of Loki care about.
1. Your PR documents upgrading steps under `docs/sources/upgrading/_index.md` if it changes:
1. Your PR documents upgrading steps under `docs/sources/setup/upgrade/_index.md` if it changes:
* Default configuration values
* Metric names or label names
* Changes existing log lines that may be used in dashboard or alerts. e.g: logs lines in any `metrics.go` files might be used in building dashboards or alerts.
Expand Down Expand Up @@ -135,7 +135,7 @@ import (

We're glad you're here to help make our technical documentation even better for Loki users.

The Grafana docs team has created a [Writer's Toolkit](https://grafana.com/docs/writers-toolkit/writing-guide/contribute-documentation/) that includes information about how we write docs, a [Style Guide](https://grafana.com/docs/writers-toolkit/style-guide/), and templates to help you contribute to the Loki documentation.
The Grafana docs team has created a [Writer's Toolkit](https://grafana.com/docs/writers-toolkit/) that includes information about how we write docs, a [Style Guide](https://grafana.com/docs/writers-toolkit/write/style-guide/), and templates to help you contribute to the Loki documentation.

The Loki documentation is written using the CommonMark flavor of markdown, including some extended features. For more information about markdown, you can see the [CommonMark specification](https://spec.commonmark.org/), and a [quick reference guide](https://commonmark.org/help/) for CommonMark.

Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -631,8 +631,8 @@ loki-operator-push: loki-operator-image-cross

documentation-helm-reference-check:
@echo "Checking diff"
$(MAKE) -BC docs sources/installation/helm/reference.md
@git diff --exit-code -- docs/sources/installation/helm/reference.md || (echo "Please generate Helm Chart reference by running 'make -C docs sources/installation/helm/reference.md'" && false)
$(MAKE) -BC docs sources/setup/install/helm/reference.md
@git diff --exit-code -- docs/sources/setup/install/helm/reference.md || (echo "Please generate Helm Chart reference by running 'make -C docs sources/setup/install/helm/reference.md'" && false)

########
# Misc #
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,13 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver

## Getting started

* [Installing Loki](https://grafana.com/docs/loki/latest/installation/)
* [Installing Loki](https://grafana.com/docs/loki/latest/setup/install/)
* [Installing Promtail](https://grafana.com/docs/loki/latest/clients/promtail/installation/)
* [Getting Started](https://grafana.com/docs/loki/latest/getting-started/)

## Upgrading

* [Upgrading Loki](https://grafana.com/docs/loki/latest/upgrading/)
* [Upgrading Loki](https://grafana.com/docs/loki/latest/setup/upgrade/)

## Documentation

Expand All @@ -44,16 +44,16 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver

Commonly used sections:

- [API documentation](https://grafana.com/docs/loki/latest/api/) for getting logs into Loki.
- [Labels](https://grafana.com/docs/loki/latest/getting-started/labels/)
- [API documentation](https://grafana.com/docs/loki/latest/reference/api/) for getting logs into Loki.
- [Labels](https://grafana.com/docs/loki/latest/get-started/labels/)
- [Operations](https://grafana.com/docs/loki/latest/operations/)
- [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) is an agent which tails log files and pushes them to Loki.
- [Pipelines](https://grafana.com/docs/loki/latest/clients/promtail/pipelines/) details the log processing pipeline.
- [Docker Driver Client](https://grafana.com/docs/loki/latest/clients/docker-driver/) is a Docker plugin to send logs directly to Loki from Docker containers.
- [LogCLI](https://grafana.com/docs/loki/latest/query/logcli/) provides a command-line interface for querying logs.
- [Loki Canary](https://grafana.com/docs/loki/latest/operations/loki-canary/) monitors your Loki installation for missing logs.
- [Troubleshooting](https://grafana.com/docs/loki/latest/getting-started/troubleshooting/) presents help dealing with error messages.
- [Loki in Grafana](https://grafana.com/docs/loki/latest/getting-started/grafana/) describes how to set up a Loki datasource in Grafana.
- [Troubleshooting](https://grafana.com/docs/loki/latest/operations/troubleshooting/) presents help dealing with error messages.
- [Loki in Grafana](https://grafana.com/docs/loki/latest/visualize/grafana/) describes how to set up a Loki datasource in Grafana.

## Getting Help

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/operations/storage/wal.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ After hitting the endpoint for `ingester-2 ingester-3`, scale down the ingesters

### Kubernetes hacking

Statefulsets are significantly more cumbersome to work with/upgrade/etc. Much of this stems from immutable fields on the specification. For example, if one wants to start using the WAL with single store Loki and wants separate volume mounts for the WAL and the boltdb-shipper, you may see immutability errors when attempting updates the Kubernetes statefulsets.
Statefulsets are significantly more cumbersome to work with, upgrade, and so on. Much of this stems from immutable fields on the specification. For example, if one wants to start using the WAL with single store Loki and wants separate volume mounts for the WAL and the boltdb-shipper, you may see immutability errors when attempting updates the Kubernetes statefulsets.

In this case, try `kubectl -n <namespace> delete sts ingester --cascade=false`. This will leave the pods alive but delete the statefulset. Then you may recreate the (updated) statefulset and one-by-one start deleting the `ingester-0` through `ingester-n` pods _in that order_, allowing the statefulset to spin up new pods to replace them.

Expand Down
1 change: 1 addition & 0 deletions docs/sources/release-notes/next.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: V?.?
description: Version ?.? release notes
weight: 100000
draft: true
---

# V?.?
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/release-notes/v2-3.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Lastly several useful additions to the LogQL query language have been included:

## Upgrade considerations

The path from 2.2.1 to 2.3.0 should be smooth, as always, read the [Upgrade Guide](https://github.com/grafana/loki/blob/master/docs/sources/upgrading/_index.md#230) for important upgrade guidance.
The path from 2.2.1 to 2.3.0 should be smooth, as always, read the [Upgrade Guide]({{< relref "../setup/upgrade#230" >}}) for important upgrade guidance.

One change we consider noteworthy however is:

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/release-notes/v2-4.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ For a full list of all changes, look at the [CHANGELOG](https://github.com/grafa

## Upgrade Considerations

Please read the [upgrade guide]({{< relref "../upgrading#240" >}}) before updating Loki.
Please read the [upgrade guide]({{< relref "../setup/upgrade#240" >}}) before updating Loki.

We made a lot of changes to Loki’s configuration as part of this release.
We have tried our best to make sure changes are compatible with existing configurations, however some changes to default limits may impact users who didn't have values explicitly set for these limits in their configuration files.
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/release-notes/v2-5.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ For a full list of all changes, look at the [CHANGELOG](https://github.com/grafa

## Upgrade Considerations

As always, please read the [upgrade guide]({{< relref "../upgrading#250" >}}) before upgrading Loki.
As always, please read the [upgrade guide]({{< relref "../setup/upgrade#250" >}}) before upgrading Loki.

### Changes to the config `split_queries_by_interval`
The most likely impact many people will see is Loki failing to start because of a change in the YAML configuration for `split_queries_by_interval`. It was previously possible to define this value in two places.
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/release-notes/v2-6.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ For a full list of all changes, look at the [CHANGELOG](https://github.com/grafa

## Upgrade Considerations

As always, please read the [upgrade guide]({{< relref "../upgrading#260" >}}) before upgrading Loki.
As always, please read the [upgrade guide]({{< relref "../setup/upgrade#260" >}}) before upgrading Loki.

## Bug fixes

Expand Down
4 changes: 2 additions & 2 deletions docs/sources/release-notes/v2-7.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Grafana Labs is excited to announce the release of Loki 2.7. Here's a summary of
- **Better Support for Azure Blob Storage** thanks to the ability to use Azure's Service Principal Credentials.
- **Logs can now be pushed from the Loki canary** so you don't have to rely on a scraping service to use the canary.
- **Additional `label_format` fields** `__timestamp__` and `__line__`.
- **`fifocache` has been renamed** The in-memory `fifocache` has been renamed to `embedded-cache`. Check [upgrade guide]({{< relref "../upgrading#270" >}}) for more details
- **`fifocache` has been renamed** The in-memory `fifocache` has been renamed to `embedded-cache`. Check [upgrade guide]({{< relref "../setup/upgrade#270" >}}) for more details
- **New HTTP endpoint for Ingester shutdown** that will also delete the ring token.
- **Faster label queries** thanks to new parallization.
- **Introducing Stream Sharding** an experimental new feature to help deal with very large streams.
Expand All @@ -30,7 +30,7 @@ For a full list of all, look at the [CHANGELOG](https://github.com/grafana/loki/

## Upgrade Considerations

As always, please read the [upgrade guide]({{< relref "../upgrading#270" >}}) before upgrading Loki.
As always, please read the [upgrade guide]({{< relref "../setup/upgrade#270" >}}) before upgrading Loki.

## Bug fixes

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/release-notes/v2-8.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ For a full list of all changes, look at the [CHANGELOG](https://github.com/grafa

## Upgrade Considerations

As always, please read the [upgrade guide]({{< relref "../upgrading#270" >}}) before upgrading Loki.
As always, please read the [upgrade guide]({{< relref "../setup/upgrade#270" >}}) before upgrading Loki.

## Bug fixes

Expand Down
13 changes: 13 additions & 0 deletions docs/sources/setup/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
title: Setup Loki
menuTitle: Setup
description: How to install and upgrade Loki, and how to migrate configurations.
weight: 300
JStickler marked this conversation as resolved.
Show resolved Hide resolved
---

# Setup Loki

- Estimate the initial [size]({{< relref "./size" >}}) for your Loki cluster.
- [Install]({{< relref "./install" >}}) Loki.
- [Migrate]({{< relref "./migrate" >}}) from one Loki implementation to another.
- [Upgrade]({{< relref "./upgrade" >}}) from one Loki version to a newer version.
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: Installation
description: Installation
menuTitle: Install
description: Describes how to install Loki.
aliases:
- ../installation/
weight: 200
---

Expand All @@ -10,11 +13,11 @@ There are several methods of installing Loki and Promtail:

- [Install using Helm (recommended)]({{< relref "./helm" >}})
- [Install using Tanka]({{< relref "./tanka" >}})
- [Install through Docker or Docker Compose]({{< relref "./docker" >}})
- [Install using Docker or Docker Compose]({{< relref "./docker" >}})
- [Install and run locally]({{< relref "./local" >}})
- [Install from source]({{< relref "./install-from-source" >}})

The [Sizing Tool]({{< relref "./sizing" >}}) can be used to determine the proper cluster sizing
The [Sizing Tool]({{< relref "../size" >}}) can be used to determine the proper cluster sizing
given an expected ingestion rate and query performance. It targets the Helm
installation on Kubernetes.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
---
title: Install Grafana Loki with Docker or Docker Compose
description: Docker
weight: 30
title: Install Loki with Docker or Docker Compose
menuTitle: Install using Docker
description: Describes how to install Loki using Docker or Docker Compose
aliases:
- ../../installation/docker/
weight: 400
---
# Install Grafana Loki with Docker or Docker Compose
# Install Loki with Docker or Docker Compose

You can install Grafana Loki and Promtail with Docker or Docker Compose if you are evaluating, testing, or developing Loki.
You can install Loki and Promtail with Docker or Docker Compose if you are evaluating, testing, or developing Loki.
For production, we recommend installing with Tanka or Helm.

The configuration acquired with these installation instructions run Loki as a single binary.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ title: Install Grafana Loki with Helm
menuTitle: Install using Helm
description: Learn how to install Grafana Loki on Kubernetes with Helm.
aliases:
- /docs/installation/helm
weight: 100
- ../../installation/helm/
JStickler marked this conversation as resolved.
Show resolved Hide resolved
weight: 200
keywords:
- helm
- scalable
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Helm chart components
menuTitle: Helm chart components
description: A short introduction to the components installed with the Loki Helm Chart.
aliases:
- /docs/installation/helm/concepts
- ../../../installation/helm/concepts/
weight: 100
keywords:
- dashboards
Expand All @@ -25,7 +25,7 @@ This chart includes dashboards for monitoring Loki. These require the scrape con

## Canary

This chart installs the [canary]({{< relref "../../operations/loki-canary" >}}) and its alerts by default. This is another tool to verify the Loki deployment is in a healthy state. It can be disabled with `monitoring.lokiCanary.enabled=false`.
This chart installs the [canary]({{< relref "../../../operations/loki-canary" >}}) and its alerts by default. This is another tool to verify the Loki deployment is in a healthy state. It can be disabled with `monitoring.lokiCanary.enabled=false`.

## Gateway

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ title: Configure storage
menuTitle: Configure storage
description: Configure Loki storage
aliases:
- /docs/installation/helm/storage
- ../../../installation/helm/configure-storage/
- ../../../installation/helm/storage
weight: 500
keywords:
- object store
Expand Down Expand Up @@ -37,7 +38,7 @@ This guide assumes Loki will be installed in one of the modes above and that a `

**To grant access to S3 via an IAM role without providing credentials:**

1. Provision an IAM role, policy and S3 bucket as described in [Storage]({{< relref "../../../storage#aws-deployment-s3-single-store" >}}).
1. Provision an IAM role, policy and S3 bucket as described in [Storage]({{< relref "../../../../storage#aws-deployment-s3-single-store" >}}).
- If the Terraform module was used note the annotation emitted by `terraform output -raw annotation`.

2. Add the IAM role annotation to the service account in `values.yaml`:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ title: Install the monolithic Helm chart
menuTitle: Install monolithic Loki
description: Install Loki in monolithic, single binary mode.
aliases:
- /docs/installation/helm/monolithic
weight: 300
keywords:
- ../../../installation/helm/monolithic
- ../../../installation/helm/install-monolithic/
weight: 100
---

# Install the monolithic Helm chart

This Helm Chart installation runs the Grafana Loki *single binary* within a Kubernetes cluster.

If you set the `singleBinary.replicas` value to 1, this chart configures Loki to run the `all` target in a [monolithic mode]({{< relref "../../../get-started/deployment-modes#monolithic-mode" >}}), designed to work with a filesystem storage. It will also configure meta-monitoring of metrics and logs.
If you set the `singleBinary.replicas` value to 1, this chart configures Loki to run the `all` target in a [monolithic mode]({{< relref "../../../../get-started/deployment-modes#monolithic-mode" >}}), designed to work with a filesystem storage. It will also configure meta-monitoring of metrics and logs.
If you set the `singleBinary.replicas` value to 2 or more, this chart configures Loki to run a *single binary* in a replicated, highly available mode. When running replicas of a single binary, you must configure object storage.

**Before you begin: Software Requirements**
Expand Down
Loading