Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Move all the config for various ways of deploying Loki into one place. (
#125) * Move all the config for various ways of deploying Loki into one place. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Fix reference to consul ksonnet module. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Move getting started docs into the production dir. Remove the raw docker commands, only use the compose file. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Spell production correctly. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Tweaks Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Tweak docs and docker compose setup; put config in images. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * typo Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
- Loading branch information
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,6 @@ | ||
| FROM alpine:3.4 | ||
| RUN apk add --update --no-cache ca-certificates && rm /var/cache/apk/* | ||
| COPY loki /bin/loki | ||
| COPY loki-local-config.yaml /etc/loki/local-config.yaml | ||
| EXPOSE 80 | ||
| ENTRYPOINT [ "/bin/loki" ] |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,6 @@ | ||
| FROM alpine:3.4 | ||
| RUN apk add --update --no-cache ca-certificates && rm /var/cache/apk/* | ||
| ADD promtail /usr/bin | ||
| COPY promtail-local-config.yaml /etc/promtail/local-config.yaml | ||
| COPY promtail-docker-config.yaml /etc/promtail/docker-config.yaml | ||
| ENTRYPOINT ["/usr/bin/promtail"] |
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
|
|
||
| ## Configuring the Loki Datasource in Grafana | ||
|
|
||
| Grafana ships with built-in support for Loki in the [latest nightly builds](https://grafana.com/grafana/download). Loki support will be officially released in Grafana 6.0. | ||
|
|
||
| 1. Open the side menu by clicking the Grafana icon in the top left. | ||
| 2. In the side menu on the left, under the Setting link you should find a link named Data Sources. | ||
| 3. Click the `+ Add data source` button in the top header. | ||
| 4. Choose Loki from the list. | ||
| 5. The http URL field should be the address of your Loki server e.g. `http://localhost:3100` or `http://loki:3100` when running with docker and docker-compose. | ||
| 6. To see the logs, click "Explore" on the sidebar, select the Loki datasource, and then choose a log stream using the "Log labels" button. | ||
|
|
||
| Read more about the Explore feature in the [Grafana docs](http://docs.grafana.org/features/explore) and on how to search and filter logs with Loki. | ||
|
|
||
| ## Grafana Provisioning | ||
|
|
||
| It is possible to configure Grafana datasources using config files with Grafana’s provisioning system. You can read more about how it works in the [Grafana documentation](http://docs.grafana.org/administration/provisioning/#datasources). | ||
|
|
||
| Here is a simple example of the provisioning yaml config for the Grafana Loki datasource: | ||
|
|
||
| ```yaml | ||
| apiVersion: 1 | ||
|
|
||
| datasources: | ||
| - name: Loki | ||
| type: loki | ||
| access: proxy | ||
| url: http://localhost:3100 | ||
| editable: false | ||
| ``` | ||
| Example with basic auth: | ||
| ```yaml | ||
| apiVersion: 1 | ||
|
|
||
| datasources: | ||
| - name: Loki | ||
| type: loki | ||
| access: proxy | ||
| url: http://localhost:3100 | ||
| editable: false | ||
| basicAuth: true | ||
| basicAuthUser: my_user | ||
| basicAuthPassword: test_password | ||
| ``` | ||
| Make sure to adjust the url and authentication to your needs, the `url` should be: | ||
|
|
||
| - `http://localhost:3100` when run Loki locally | ||
| - `http://loki:3100` when run Loki with docker-compose | ||
|
|
||
| `basicAuthUser` and `basicAuthPassword` should same as your Grafana setting. | ||
|
|
||
| ## Searching with Labels and Distributed Grep | ||
|
|
||
| A log query consists of two parts: **log stream selector**, and a **search expression**. For performance reasons you need to start by choosing a log stream by selecting a log label. | ||
|
|
||
| The log stream selector will reduce the number of log streams to a manageable volume and then the regex search expression is used to do a distributed grep over those log streams. | ||
|
|
||
| Searching can be done in the Explore section of Grafana (latest nightly builds) or via the `logcli` tool which is documented [here](https://github.com/grafana/loki/blob/master/docs/logcli.md). | ||
|
|
||
| ### Log Stream Selector | ||
|
|
||
| For the label part of the query expression, wrap it in curly braces `{}` and then use the key value syntax for selecting labels. Multiple label expressions are separated by a comma: | ||
|
|
||
| `{app="mysql",name="mysql-backup"}` | ||
|
|
||
| The following label matching operators are currently supported: | ||
|
|
||
| - `=` exactly equal. | ||
| - `!=` not equal. | ||
| - `=~` regex-match. | ||
| - `!~` do not regex-match. | ||
|
|
||
| Examples: | ||
|
|
||
| - `{name=~"mysql.+"}` | ||
| - `{name!~"mysql.+"}` | ||
|
|
||
| The [same rules that apply for Prometheus Label Selectors](https://prometheus.io/docs/prometheus/latest/querying/basics/#instant-vector-selectors) apply for Loki Log Stream Selectors. | ||
|
|
||
| ### Regex Search Expression | ||
|
|
||
| After writing the Log Stream Selector, you can filter the results further by writing a search expression. The search expression can be just text or a regex expression. | ||
|
|
||
| Example queries: | ||
|
|
||
| - `{job="mysql"} error` | ||
| - `{name="kafka"} tsdb-ops.*io:2003` | ||
| - `{instance=~"kafka-[23]",name="kafka"} kafka.server:type=ReplicaManager` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # Running Loki | ||
|
|
||
| Currently there are four ways to try out Loki, in order from easier to hardest: | ||
|
|
||
| - [Using our free hosted demo](#free-hosted-demo) | ||
| - [Running it locally with Docker](#run-locally-using-docker) | ||
| - [Building from source](#build-and-run-from-source) | ||
| - [Using our Ksonnet config to run a fully-blown production setup](ksonnet/) | ||
|
|
||
| ## Free Hosted Demo | ||
|
|
||
| Grafana is running a free, hosted demo cluster of Loki; instructions for getting access can be found at [grafana.com](https://grafana.com/loki). | ||
|
|
||
| ## Run Locally Using Docker | ||
|
|
||
| The Docker images for [Loki](https://hub.docker.com/r/grafana/loki/) and [Promtail](https://hub.docker.com/r/grafana/promtail/) are available on DockerHub. | ||
|
|
||
| To test locally, we recommend using the docker-compose.yaml file in this directory: | ||
|
|
||
| 1. Either `git clone` this repository locally and `cd loki/production`, or download a copy of the [docker-compose.yaml](docker-compose.yaml) locally. | ||
|
|
||
| 1. Ensure you have the freshest, most up to date container images: | ||
|
|
||
| ```bash | ||
| docker-compose pull | ||
| ``` | ||
|
|
||
| 1. Run the stack on your local docker: | ||
|
|
||
| ```bash | ||
| docker-compose up | ||
| ``` | ||
|
|
||
| 1. Grafana should now be available at http://localhost:3000/. Follow the [steps for configuring the datasource in Grafana](../docs/usage.md) and set the URL field to `http://loki:3100`. | ||
|
|
||
| For instructions on how to use loki, see [our usage docs](../docs/usage.md). | ||
|
|
||
| ## Build and Run From Source | ||
|
|
||
| Loki can be run in a single host, no-dependencies mode using the following commands. | ||
|
|
||
| You need `go` [v1.10+](https://golang.org/dl/) installed locally. | ||
|
|
||
| ```bash | ||
| $ go build ./cmd/loki | ||
| $ ./loki -config.file=./cmd/loki/loki-local-config.yaml | ||
| ... | ||
| ``` | ||
|
|
||
| To run Promtail, use the following commands: | ||
|
|
||
| ```bash | ||
| $ go build ./cmd/promtail | ||
| $ ./promtail -config.file=./cmd/promtail/promtail-local-config.yaml | ||
| ... | ||
| ``` | ||
|
|
||
| Grafana is Loki's UI, so you'll also want to run one of those: | ||
|
|
||
| ```bash | ||
| $ docker run -ti -p 3000:3000 -e "GF_EXPLORE_ENABLED=true" grafana/grafana:master | ||
| ``` | ||
|
|
||
| Grafana should now be available at http://localhost:3000/. Follow the [steps for configuring the datasource in Grafana](../docs/usage.md) and set the URL field to `http://host.docker.internal:3100`. | ||
|
|
||
| For instructions on how to use loki, see [our usage docs](../docs/usage.md). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Hi, when i start: "docker-compose up" then start it loki, grafana & promtail but after a few seconds promtail stops. error:
production_promtail_1 exited with code 137