A YAML-defined Custom Prometheus exporter

Create your own Prometheus exporters using simple YAML.

This project allows you to create your own Prometheus exporter which will generate and publish the metrics of your choice in the Prometheus format. All this through a simple YAML configuration file. The goal of this project is to allow you to quickly create and easily augment a new Prometheus exporter without having to know how to write a native exporter.

Using a short YAML configuration file, you can define your own metrics and make them available for Prometheus to scrape.

Using the Custom Prometheus Exporter

The simplest way to try out the Custom Prometheus Exporter is through docker:

docker run --rm \
    --name custom-prometheus-exporter -p 12345:12345 \
    -v $(pwd)/example-configurations/test-exporter.yaml:/tmp/test-exporter.yaml \
    marckhouzam/custom-prometheus-exporter -f /tmp/test-exporter.yaml

Then you can see the metrics using:

curl localhost:12345/test

Configuration

The Custom Prometheus Exporter take one or more YAML-configuration files, which specify the metrics that are to be collected and how to collect them. Example configurations can be found in the directory example-configurations.

Here is a sample configuration. This configuration will create a "docker-exporter", which can be scraped on port 9550 and endpoint /metrics. This exporter generates a single metric (named: docker_container_states_containers) that provides the count of containers in their three possible states (Running, Stopped, Paused). This metric is collected on each call to the /metrics endpoint using the three sh-shell commands specified in executions.

name: docker-exporter
port: 9550
endpoint: /metrics
metrics:
- name: docker_container_states_containers
  help: The count of containers in various states
  type: gauge
  executions:
  - type: sh
    command: docker info --format '{{ .ContainersRunning }}'
    labels:
      state: Running
  - type: sh
    command: docker info --format '{{ .ContainersStopped }}'
    labels:
      state: Stopped
  - type: sh
    command: docker info --format '{{ .ContainersPaused }}'
    labels:
      state: Paused

The generated metric looks like this:

# HELP docker_container_states_containers The count of containers in various states
# TYPE docker_container_states_containers gauge
docker_container_states_containers{state="Paused"} 0
docker_container_states_containers{state="Running"} 0
docker_container_states_containers{state="Stopped"} 4

Multiple exporters

The Custom Prometheus Exporter allows you to define many exporters at once. Each exporter must be in its own YAML configuration file. All defined exporters will be run concurrently and be accessible using their own configuration-specified port and endpoint. If you want to create metrics that are logically different, it is recommended to use multiple exporters instead of a single exporter lumping all the unrelated metrics together. Besides cleanly separating the definition of each logical exporter, the separation also allows each exporter can be scraped at different intervals.

You may instead choose to run the Custom Prometheus Exporter multiple times, one for each exporter you want to create. This choice is entirely up to you.

Here is how to run both example exporters together, using Docker:

docker run --rm \
    --name custom-prometheus-exporter -p 12345:12345 -p 9550:9550 \
    -v $(pwd)/example-configurations/test-exporter.yaml:/tmp/test-exporter.yaml \
    -v $(pwd)/example-configurations/docker-exporter.yaml:/tmp/docker-exporter.yaml \
    -v /var/run/docker.sock:/var/run/docker.sock \
    marckhouzam/custom-prometheus-exporter -f /tmp/test-exporter.yaml -f /tmp/docker-exporter.yaml

Then you can see the metrics using:

curl localhost:9550/metrics
curl localhost:12345/test

Note that the example docker-exporter.yaml uses docker commands. To be able to run docker commands inside a docker container, you must mount /var/run/docker.sock as shown above.

Main Custom Prometheus Exporter endpoints

The actual Custom Prometheus Exporter provides its own endpoints. By default, the Custom Prometheus Exporter listens on port 9530 but it can be changed using the -p command-line parameter. This port is not related to the exporters you define, but only to the global Custom Prometheus Exporter endpoints.

You can obtain a list of main endpoints by navigating to http://localhost:9530.

Configuration API

The format of the YAML configuration is the following:

name: string          # A name for the exporter - MANDATORY
port: int             # The TCP port serving the metrics - MANDATORY
endpoint: string      # The endpoint serving the metrics - OPTIONAL, defaults to /metrics
metrics:              # An array of metrics to be generated - MANDATORY
- name: string        # The published name of the metric - MANDATORY
  help: string        # The published help message of the metric - MANDATORY
  type: gauge         # Only Prometheus "gauge" is currently supported - MANDATORY
  executions:         # An array of executions to generate the metric - MANDATORY
  - type: sh          # Only sh is currently supported - MANDATORY
    command: string   # An sh command that will be run exactly as-specified - MANDATORY
                      #   Shell pipes (|) are allowed.
                      #   The result of the command must be the single
                      #      integer to be used in the metric
    labels: map(string, string)
                      # A map of label to value which qualifies an instance - MANDATORY
                      #   of the metric

Backwards-compatibility considerations

Once your YAML-defined exporter is being used, you should be careful when making modifications to its YAML-definition. It may seem harmless to change the configuration, but changes to some fields could cause consumers to break (such as Prometheus alerts, or Grafana dashboards).

Below are the fields that you should treat as API:

name: string
port: int             # API - Changes affect Prometheus configuration
endpoint: string      # API - Changes affect Prometheus configuration
metrics:
- name: string        # API - Changes affect consumers of metrics
  help: string
  type: gauge
  executions:
  - type: sh
    command: string
    labels: map(string, string)
                      # API - Changes affect consumers of metrics

Contributing

Contributions are welcomed! You can submit code or documentation Pull Requests, or open issues with ideas or problems you found.

Building the project

The code is written in Go. If you don't already have your Go development environment setup, start here.

To compile the custom-prometheus-exporter you will also need to install the go yaml package:

go get gopkg.in/yaml.v2

Then get and compile the code:

git clone https://github.com/marckhouzam/custom-prometheus-exporter.git
cd custom-prometheus-exporter
go build

Running

Natively, after you've compiled it:

./custom-prometheus-exporter -f yamlConfigFile1 [-f yamlConfigFile2] ...

Docker

You an also use Docker:

docker build -t custom-prometheus-exporter .
docker run --rm -d \
    --name custom-prometheus-exporter -p <port>:<yourExporterPort> \
    -v <yourExporterConfigFile.yaml>:/tmp/exporter.yaml \
    custom-prometheus-exporter -f /tmp/exporterConfig.yaml

TODO list

• Automated Tests
• Support other types of metrics (e.g., Counter)
• Complete /-/reload support
• Support for bash and other shells
• Support for native execution instead of shell command (e.g., running a script)
• Add a Kubernetes Helm chart