Skip to content

Commit

Permalink
more informative logging, code formatting, readme restructured
Browse files Browse the repository at this point in the history
  • Loading branch information
@or-else
or-else committed Apr 3, 2020
1 parent 00cdd35 commit 4bde839
Show file tree
Hide file tree
Showing 3 changed files with 87 additions and 51 deletions.
59 changes: 32 additions & 27 deletions monitoring/exporter/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
# Tinode Metric Exporter

This is a simple service which reads JSON monitoring data exposed by Tinode server using [expvar](https://golang.org/pkg/expvar/) and re-publishes it in other formats.
Currently, supported are:
* [Prometheus](https://prometheus.io/) [exporter](https://prometheus.io/docs/instrumenting/exporters/) which exports data in [prometheus format](https://prometheus.io/docs/concepts/data_model/).
Note that the monitoring service is expected to pull/scrape data from Prometheus exporter.
* [InfluxDB](https://www.influxdata.com/) [exporter](https://docs.influxdata.com/influxdb/v1.7/tools/api/#write-http-endpoint), on the contrary, pushes data to its target backend.
This is a simple service which reads JSON monitoring data exposed by Tinode server using [expvar](https://golang.org/pkg/expvar/) and re-publishes it in other formats. Currently the supported formats are:

* [InfluxDB](https://www.influxdata.com/) [exporter](https://docs.influxdata.com/influxdb/v1.7/tools/api/#write-http-endpoint) **pushes** data to its target backend. This is the default mode.
* [Prometheus](https://prometheus.io/) [exporter](https://prometheus.io/docs/instrumenting/exporters/) which exports data in [prometheus format](https://prometheus.io/docs/concepts/data_model/). The Prometheus monitoring service is expected to **pull/scrape** data from the exporter.

## Usage

Exporters are expected to run next to (pair with) Tinode servers: one Exporter per one Tinode server, i.e. a single Exporter provides metrics from a single Tinode server.
Currently, the Exporter is fully configured via command line flags. There are three sets of flags:
Exporters are intended to run next to (pair with) Tinode servers: one Exporter per one Tinode server, i.e. a single Exporter provides metrics from a single Tinode server.

## Configuration

The exporters are configured by command-line flags:

### Common flags
* `serve_for` specifies which monitoring service the Exporter will gather metrics for; accepted values: `influxdb`, `prometheus`; default: `influxdb`.
Expand All @@ -18,11 +20,6 @@ Currently, the Exporter is fully configured via command line flags. There are th
* `instance` is the Exporter instance name (it may be exported to the upstream backend); default: `exporter`.
* `metric_list` is a comma-separated list of metrics to export; default: `Version,LiveTopics,TotalTopics,LiveSessions,ClusterLeader,TotalClusterNodes,LiveClusterNodes,memstats.Alloc`.

### Prometheus
* `prom_namespace` is a prefix to use for metrics names. If you are monitoring multiple tinode instances you may want to use different namespaces; default: `tinode`.
* `prom_metrics_path` is the path under which to expose the metrics for scraping; default: `/metrics`.
* `prom_timeout` is the Tinode connection timeout in seconds in response to Prometheus scrapes; default: `15`.

### InfluxDB
* `influx_push_addr` is the address of InfluxDB target server where the data gets sent; default: `http://localhost:9999/write`.
* `influx_db_version` is the version of InfluxDB (only 1.7 and 2.0 are supported); default: `1.7`.
Expand All @@ -31,21 +28,7 @@ Currently, the Exporter is fully configured via command line flags. There are th
* `influx_auth_token` - InfluxDB authentication token; no default value.
* `influx_push_interval` - InfluxDB push interval in seconds; default: `30`.

## Examples
Run Prometheus Exporter as
```
./exporter \
--serve_for=prometheus \
--tinode_addr=http://localhost:6060/stats/expvar \
--listen_at=:6222 \
--instance=exp-0 \
--prom_namespace=tinode \
--prom_metrics_path=/metrics \
--prom_timeout=15
```

This exporter will serve data at path /metrics, on port 6222.
Once running, configure your Prometheus monitoring installation to collect data from this exporter.
#### Example

Run InfluxDB Exporter as
```
Expand All @@ -62,3 +45,25 @@ Run InfluxDB Exporter as
```

This exporter will push the collected metrics to the specified backend once every 30 seconds.


### Prometheus
* `prom_namespace` is a prefix to use for metrics names. If you are monitoring multiple tinode instances you may want to use different namespaces; default: `tinode`.
* `prom_metrics_path` is the path under which to expose the metrics for scraping; default: `/metrics`.
* `prom_timeout` is the Tinode connection timeout in seconds in response to Prometheus scrapes; default: `15`.

#### Example
Run Prometheus Exporter as
```
./exporter \
--serve_for=prometheus \
--tinode_addr=http://localhost:6060/stats/expvar \
--listen_at=:6222 \
--instance=exp-0 \
--prom_namespace=tinode \
--prom_metrics_path=/metrics \
--prom_timeout=15
```

This exporter will serve data at path /metrics, on port 6222.
Once running, configure your Prometheus monitoring installation to collect data from this exporter.
27 changes: 21 additions & 6 deletions monitoring/exporter/influxdb_exporter.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
"strings"
"time"
)

Expand All @@ -20,7 +22,9 @@ type InfluxDBExporter struct {
}

// NewInfluxDBExporter returns an initialized InfluxDB exporter.
func NewInfluxDBExporter(influxDBVersion, pushBaseAddress, organization, bucket, token, instance string, scraper *Scraper) *InfluxDBExporter {
func NewInfluxDBExporter(influxDBVersion, pushBaseAddress, organization,
bucket, token, instance string, scraper *Scraper) *InfluxDBExporter {

targetAddress := formPushTargetAddress(influxDBVersion, pushBaseAddress, organization, bucket)
tokenHeader := formAuthorizationHeaderValue(influxDBVersion, token)
return &InfluxDBExporter{
Expand All @@ -34,10 +38,10 @@ func NewInfluxDBExporter(influxDBVersion, pushBaseAddress, organization, bucket,
}

// Push scrapes metrics from Tinode server and pushes these metrics to InfluxDB.
func (e *InfluxDBExporter) Push() (string, error) {
func (e *InfluxDBExporter) Push() error {
metrics, err := e.scraper.CollectRaw()
if err != nil {
return "", err
return err
}
b := new(bytes.Buffer)
ts := time.Now().UnixNano()
Expand All @@ -46,15 +50,26 @@ func (e *InfluxDBExporter) Push() (string, error) {
}
req, err := http.NewRequest("POST", e.targetAddress, b)
if err != nil {
return "", err
return err
}
req.Header.Add("Authorization", e.tokenHeader)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return "", err
return err
}
defer resp.Body.Close()
return resp.Status, nil

if resp.StatusCode >= 400 {
var body string
if rb, err := ioutil.ReadAll(resp.Body); err != nil {
body = err.Error()
} else {
body = strings.TrimSpace(string(rb))
}

return fmt.Errorf("HTTP %s: %s", resp.Status, body)
}
return nil
}

func formPushTargetAddress(influxDBVersion, baseAddr, organization, bucket string) string {
Expand Down
52 changes: 34 additions & 18 deletions monitoring/exporter/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,24 +45,36 @@ func main() {
log.Printf("Tinode metrics exporter.")

var (
serveFor = flag.String("serve_for", "influxdb", "Monitoring service to gather metrics for. Available: influxdb, prometheus.")
tinodeAddr = flag.String("tinode_addr", "http://localhost:6060/stats/expvar", "Address of the Tinode instance to scrape.")
listenAt = flag.String("listen_at", ":6222", "Host name and port to listen for incoming requests on.")
metricList = flag.String("metric_list", "Version,LiveTopics,TotalTopics,LiveSessions,ClusterLeader,TotalClusterNodes,LiveClusterNodes,memstats.Alloc", "Comma-separated list of metrics to scrape and export.")
instance = flag.String("instance", "exporter", "Exporter instance name.")
serveFor = flag.String("serve_for", "influxdb",
"Monitoring service to gather metrics for. Available: influxdb, prometheus.")
tinodeAddr = flag.String("tinode_addr", "http://localhost:6060/stats/expvar",
"Address of the Tinode instance to scrape.")
listenAt = flag.String("listen_at", ":6222",
"Host name and port to listen for incoming requests on.")
metricList = flag.String("metric_list",
"Version,LiveTopics,TotalTopics,LiveSessions,ClusterLeader,TotalClusterNodes,LiveClusterNodes,memstats.Alloc",
"Comma-separated list of metrics to scrape and export.")
instance = flag.String("instance", "exporter",
"Exporter instance name.")

// Prometheus-specific arguments.
promNamespace = flag.String("prom_namespace", "tinode", "Prometheus namespace for metrics '<namespace>_...'")
promMetricsPath = flag.String("prom_metrics_path", "/metrics", "Path under which to expose metrics for Prometheus scrapes.")
promTimeout = flag.Int("prom_timeout", 15, "Tinode connection timeout in seconds in response to Prometheus scrapes.")

// InfluxDB-specific arguments.
influxPushAddr = flag.String("influx_push_addr", "http://localhost:9999/write", "Address of InfluxDB target server where the data gets sent.")
influxDBVersion = flag.String("influx_db_version", "1.7", "Version of InfluxDB (only 1.7 and 2.0 are supported).")
influxOrganization = flag.String("influx_organization", "test", "InfluxDB organization to push metrics as.")
influxBucket = flag.String("influx_bucket", "test", "InfluxDB storage bucket to store data in (used only in InfluxDB 2.0).")
influxAuthToken = flag.String("influx_auth_token", "", "InfluxDB authentication token.")
influxPushInterval = flag.Int("influx_push_interval", 30, "InfluxDB push interval in seconds.")
influxPushAddr = flag.String("influx_push_addr", "http://localhost:9999/write",
"Address of InfluxDB target server where the data gets sent.")
influxDBVersion = flag.String("influx_db_version", "1.7",
"Version of InfluxDB (only 1.7 and 2.0 are supported).")
influxOrganization = flag.String("influx_organization", "test",
"InfluxDB organization to push metrics as.")
influxBucket = flag.String("influx_bucket", "test",
"InfluxDB storage bucket to store data in (used only in InfluxDB 2.0).")
influxAuthToken = flag.String("influx_auth_token", "",
"InfluxDB authentication token.")
influxPushInterval = flag.Int("influx_push_interval", 30,
"InfluxDB push interval in seconds.")
)
flag.Parse()

Expand Down Expand Up @@ -91,7 +103,7 @@ func main() {
log.Fatal("Must specify --influx_bucket")
}
if *influxDBVersion != "1.7" && *influxDBVersion != "2.0" {
log.Fatal("Please, set --influx_db_version to either 1.7 or 2.0")
log.Fatal("The --influx_db_version must be either 1.7 or 2.0")
}
if *influxPushInterval > 0 && *influxPushInterval < minPushInterval {
*influxPushInterval = minPushInterval
Expand All @@ -118,6 +130,9 @@ func main() {
})

metrics := strings.Split(*metricList, ",")
for i, m := range metrics {
metrics[i] = strings.TrimSpace(m)
}
scraper := Scraper{address: *tinodeAddr, metrics: metrics}
var serverTypeString string
// Create exporters.
Expand All @@ -141,15 +156,16 @@ func main() {
)
case InfluxDB:
serverTypeString = fmt.Sprintf("%s, version %s", *serveFor, *influxDBVersion)
influxDBExporter := NewInfluxDBExporter(*influxDBVersion, *influxPushAddr, *influxOrganization, *influxBucket, *influxAuthToken, *instance, &scraper)
influxDBExporter := NewInfluxDBExporter(*influxDBVersion, *influxPushAddr, *influxOrganization, *influxBucket,
*influxAuthToken, *instance, &scraper)
if *influxPushInterval > 0 {
go func() {
interval := time.Duration(*influxPushInterval) * time.Second
ch := time.Tick(interval)
for {
if _, ok := <-ch; ok {
if status, err := influxDBExporter.Push(); err != nil {
log.Printf("InfluxDB push failed (status '%s'), error: %s", status, err.Error())
if err := influxDBExporter.Push(); err != nil {
log.Println("InfluxDB push failed:", err)
}
} else {
return
Expand All @@ -162,10 +178,10 @@ func main() {
// Forces a data push.
http.HandleFunc("/push", func(w http.ResponseWriter, r *http.Request) {
var msg string
if http_status, err := influxDBExporter.Push(); err == nil {
msg = "ok - " + http_status
if err := influxDBExporter.Push(); err == nil {
msg = "HTTP 200 OK"
} else {
msg = "fail - " + err.Error()
msg = err.Error()
}

w.Write([]byte(`<html><head><title>Tinode Push</title></head><body>
Expand Down

0 comments on commit 4bde839

Please sign in to comment.