Add Grafana integration example (#2408)

* add basic Grafana integration setup and README.md

Signed-off-by: fktkrt <fktkrt@gmail.com>

* move instructions to README, fix typo

Signed-off-by: fktkrt <fktkrt@gmail.com>

* add prometheus as datasource and hotrod scrape_config

Signed-off-by: fktkrt <fktkrt@gmail.com>

* reorg into folders, add hotROD dashboard, add basic description to README

Signed-off-by: fktkrt <fktkrt@gmail.com>

* rework folder structure, remove duplicated file, accept suggestions

Signed-off-by: fktkrt <fktkrt@gmail.com>

examples/grafana-integration/README.md

@@ -0,0 +1,55 @@
+# Hot R.O.D. - Rides on Demand  - Grafana integration
+
+This example combines the Hot R.O.D. demo application ([examples/hotrod/](../hotrod/)) with Grafana, Loki and Prometheus integration, to demonstrate logs, metrics and traces correlation.
+
+## Running via `docker-compose`
+
+### Prerequisites
+
+* Clone the Jaeger repository `git clone https://github.com/jaegertracing/jaeger.git`, then `cd examples/grafana-integration`
+
+* All services will log to Loki via the [official Docker driver plugin](https://grafana.com/docs/loki/latest/clients/docker-driver/).
+Install the Loki logging plugin for Docker:
+
+```bash
+docker plugin install \
+grafana/loki-docker-driver:latest \
+--alias loki \
+--grant-all-permissions
+```
+
+### Run the services
+
+`docker-compose up` 
+
+### Access the services
+* HotROD application at http://localhost:8080
+* Grafana UI at http://localhost:3000
+
+### Explore with Loki
+
+Currently the most powerful way to correlate application logs with traces can be performed via Grafana's Explore interface.
+
+After setting the datasource to Loki, all the log labels become available, and can be easily filtered using [Loki's LogQL query language](https://grafana.com/docs/loki/latest/logql/).
+
+For example, after selecting the compose project/service under Log labels , errors can be filtered with the following expression:
+
+```
+{compose-project="grafana-integration"} |= "error"
+```
+
+which will list the redis timeout events.
+
+### HotROD - Metrics and logs overview dashboard
+
+Since the HotROD application can expose its metrics in Prometheus' format, these can be also used during investigation.
+
+This example includes a dashboard that contains a log panel for the selected services in real time. These can be also filtered by a search field, that provides `grep`-like features.
+
+There are also panels to display the ratio/percentage of errors in the current timeframe.
+
+Additionally, there are graphs for each service, visualing the rate of the requests and showing latency percentiles.
+
+### Clean up
+
+`docker-compose down`

examples/grafana-integration/docker-compose.yaml

@@ -0,0 +1,68 @@
+version: '2'
+
+services:
+  grafana:
+    image: grafana/grafana:7.1.4
+    ports:
+      - '3000:3000'
+    volumes:
+      - ./grafana/datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
+      - ./grafana/dashboards.yml:/etc/grafana/provisioning/dashboards/dashboards.yml
+      - ./grafana/hotrod_metrics_logs.json:/etc/grafana/provisioning/dashboards/hotrod_metrics_logs.json
+    logging:
+      driver: loki
+      options:
+        loki-url: 'http://localhost:3100/api/prom/push'
+
+  loki:
+    image: grafana/loki:master
+    ports:
+      - '3100:3100'
+    command: -config.file=/etc/loki/local-config.yaml
+    # send Loki traces to Jaeger
+    environment:
+      - JAEGER_AGENT_HOST=jaeger
+      - JAEGER_AGENT_PORT=6831
+      - JAEGER_SAMPLER_TYPE=const
+      - JAEGER_SAMPLER_PARAM=1
+    logging:
+      driver: loki
+      options:
+        loki-url: 'http://localhost:3100/api/prom/push'
+
+  jaeger:
+    image: jaegertracing/all-in-one:latest
+    ports:
+      - '6831:6831'
+      - '16686:16686'
+    logging:
+      driver: loki
+      options:
+        loki-url: 'http://localhost:3100/api/prom/push'
+
+  hotrod:
+    image: jaegertracing/example-hotrod:latest
+    ports:
+      - '8080:8080'
+      - '8083:8083'
+    command: ["-m","prometheus","all"]
+    environment:
+      - JAEGER_AGENT_HOST=jaeger
+      - JAEGER_AGENT_PORT=6831
+    logging:
+      driver: loki
+      options:
+        loki-url: 'http://localhost:3100/api/prom/push'
+
+  prometheus:
+    image: prom/prometheus:v2.19.1
+    volumes:
+      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
+    ports:
+      - '9090:9090'
+    command:
+    - --config.file=/etc/prometheus/prometheus.yml
+    logging:
+      driver: loki
+      options:
+        loki-url: 'http://localhost:3100/api/prom/push'

examples/grafana-integration/grafana/dashboard.yml

@@ -0,0 +1,11 @@
+apiVersion: 1
+
+providers:
+- name: 'HotROD'
+  orgId: 1
+  folder: ''
+  type: file
+  disableDeletion: false
+  editable: true
+  options:
+    path: /etc/grafana/provisioning/dashboards

examples/grafana-integration/grafana/datasources.yaml

@@ -0,0 +1,18 @@
+apiVersion: 1
+
+datasources:
+  - name: Loki
+    type: loki
+    access: proxy
+    url: http://loki:3100
+    editable: true
+    isDefault: true
+  - name: Jaeger
+    type: jaeger
+    access: browser
+    url: http://jaeger:16686
+    editable: true
+  - name: Prometheus
+    type: prometheus
+    access: proxy
+    url: http://prometheus:9090