NGINX-to-Prometheus log file exporter

Helper tool that continuously reads an NGINX log file and exports metrics to Prometheus.

Usage

You can either use a simple configuration, using command-line flags, or create a configuration file with a more advanced configration.

Use the command-line:

./nginx-log-exporter \
  -format="<FORMAT>" \
  -listen-port=4040 \
  -namespace=nginx \
  [PATHS-TO-LOGFILES...]

Use the configuration file:

./nginx-log-exporter -config-file /path/to/config.hcl

Collected metrics

This exporter collects the following metrics. This collector can listen on multiple log files at once and publish metrics in different namespaces. Each metric uses the labels method (containing the HTTP request method) and status (containing the HTTP status code).

• <namespace>_http_response_count_total - The total amount of processed HTTP requests/responses.
• <namespace>_http_response_size_bytes - The total amount of transferred content in bytes.
• <namespace>_http_upstream_time_seconds - A summary vector of the upstream response times in seconds. Logging these needs to be specifically enabled in NGINX using the $upstream_response_time variable in the log format.
• <namespace>_http_response_time_seconds - A summary vector of the total response times in seconds. Logging these needs to be specifically enabled in NGINX using the $request_time variable in the log format.

Additional labels can be configured in the configuration file (see below).

Configuration file

You can specify a configuration file to read at startup. The configuration file is expected to be in HCL format. Here's an example file:

listen {
  port = 4040
  address = "10.1.2.3"
}

consul {
  enable = true
  address = "localhost:8500"
  service {
    id = "nginx-exporter"
    name = "nginx-exporter"
    datacenter = "dc1"
    scheme = "http"
    token = ""
    tags = ["foo", "bar"]
  }
}

namespace "app-1" {
  format = "$remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$http_x_forwarded_for\""
  source_files = [
    "/var/log/nginx/app1/access.log"
  ]
  labels {
    app = "application-one"
    environment = "production"
    foo = "bar"
  }
}

namespace "app-2" {
  format = "$remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$http_x_forwarded_for\" $upstream_response_time
  source_file = [
    "/var/log/nginx/app2/access.log"
  ]
}

Credits

• tail, MIT license
• gonx, MIT license
• Prometheus Go client library, Apache License
• HashiCorp configuration language, Mozilla Public License