Skip to content

Add stats-exporter() doc #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions _data/external_links.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1112,6 +1112,11 @@ postgresql:
url: https://www.postgresql.org/docs/current/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
title: [ "PostgreSQL" ]

prometheus-scraper:
id: prometheus-scraper
url: https://prometheus.io/docs/prometheus/latest/getting_started/#configure-prometheus-to-monitor-the-sample-targets
title: [ "/[Pp]rometheus [Ss]craper/" ]

qbittorrent:
id: qbittorrent
url: https://www.qbittorrent.org/
Expand Down
3 changes: 3 additions & 0 deletions _data/link_aliases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ adm-about-glossary#bsd-syslog-protocol:
adm-about-glossary#ietf-syslog-protocol:
aliases: [ "IETF-syslog protocol", "IETF-syslog" ]

adm-src-stats-exporter:
aliases: [ "stats-exporter-dont-log()" ]

adm-src-wild:
aliases: [ "/wildcard-file(?:\\(\\))? source[s]?/" ]

Expand Down
10 changes: 10 additions & 0 deletions _data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,11 @@ admin-guide-nav:
subnav:
- title: "snmptrap() source options"
url: /admin-guide/060_Sources/150_snmptrap/000_snmptrap_options
- title: "stats-exporter"
url: /admin-guide/060_Sources/153_stats_exporter/README
subnav:
- title: "stats-exporter() source options"
url: /admin-guide/060_Sources/153_stats_exporter/000_stats_exporter_source_options
- title: "stdin"
url: /admin-guide/060_Sources/155_stdin/README
subnav:
Expand Down Expand Up @@ -302,6 +307,11 @@ admin-guide-nav:
url: /admin-guide/060_Sources/220_unix-stream_unix-dgram/000_Unix_credentials
- title: "unix-stream() and unix-dgram() source options"
url: /admin-guide/060_Sources/220_unix-stream_unix-dgram/001_unixstream_source_options
- title: "webhook"
url: /admin-guide/060_Sources/240_webhook/README
subnav:
- title: "webhook() source options"
url: /admin-guide/060_Sources/240_webhook/000_webhook_options
- title: "Destinations"
url: /admin-guide/070_Destinations/README
subnav:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: stats-exporter() source options
id: adm-src--stats-exporter-opt
description: >-
The stats-exporter() sources directly can serve the output of syslog-ng-ctl stats and syslog-ng-ctl query to a HTTP Scraper. Unlike the stats-exporter-dont-log() that suppresses log messages from incoming scraper requests, the stats-exporter() logs the unparsed messages, storing incoming scraper HTTP requests in the MSG field.
---

Technically, both sources are specialized versions of the network() source. See the network() source options for details.

>**NOTE:** A destination is not required for this source to work; the `stats-exporter()` sources will respond to the scraper regardless of whether a destination is present in the log path.
{: .notice--info}

These drivers have the following additional options:

## stat-type()

|Accepted values:| query \| stats |
|Default: | stats |

*Description:* The stat type you wish to provide in the response to a HTTP Scraper request, just like for the syslog-ng-ctl command line tool.

## stat-query()

|Accepted values:| regular expression |
|Default: | |

*Description:* The query regex string that can be used to filter the output of a `query` type request.

## stat-format()

|Accepted values:| csv \| kv \| prometheus |
|Default: | prometheus |

*Description:* Specifies the format of the statistics output in HTTP responses, similar to the options available in syslog-ng-ctl. The available formats are:

- csv – comma-separated values format
- kv – key-value pairs, one per line
- prometheus – Prometheus-compatible exposition format

## scrape-pattern()

|Accepted values:| regular expression |
|Default: | GET /metrics* |

*Description:* The pattern used to match the HTTP header of incoming scraping requests. A stat response will be generated and sent only if the header matches the scrape-pattern().

## scrape-freq-limit()

|Accepted values:| number |
|Default: | 0 |

*Description:* Limits the frequency of repeated scraper requests to the specified number of seconds. Any repeated request within this period will be ignored. A value of 0 means no limit.

## single-instance()

|Accepted values:| yes \| no |
|Default: | no |

*Description:* If set to `yes` only one scraper connection and request will be allowed at once.
57 changes: 57 additions & 0 deletions doc/_admin-guide/060_Sources/153_stats_exporter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: 'stats-exporter: Providing various statistics information to a HTTP Scraper'
short_title: stats-exporter
id: adm-src-stats-exporter
description: >-
The stats-exporter() and stats-exporter-dont-log() sources can directly serve the output of
syslog-ng-ctl stats and syslog-ng-ctl query to an HTTP scraper, such as Prometheus.
---

Example usage for a Prometheus Scraper which logs the HTTP request of the scraper to /var/log/scraper.log:

``` config
@version: 4.9
@include "scl.conf"

source s_prometheus_stat {
stats-exporter(
ip("0.0.0.0")
port(8080)
stat-type("query")
stat-query("*")
scrape-freq-limit(30)
single-instance(yes)
);
};

log {
source(s_prometheus_stat);
destination { file(/var/log/scraper.log); };
};
```

Example usage for a generic HTTP Scraper which sends e.g. the `GET /stats HTTP/1.1` HTTP request to get statistics of {{ site.product.short_name }}, do not want to log or further process the HTTP requests in the log pipe, and needs the response in CSV format:

``` config
@version: 4.9
@include "scl.conf"

source s_scraper_stat {
stats-exporter-dont-log(
ip("0.0.0.0")
port(8080)
stat-type("stats")
stat-format("csv")
scrape-pattern("GET /stats*")
scrape-freq-limit(30)
single-instance(yes)
);
};

log {
source(s_scraper_stat);
};
```

>**NOTE:** A destination is not required for this to work; the `stats-exporter()` sources will respond to the scraper regardless of whether a destination is present in the log path.
{: .notice--info}
3 changes: 1 addition & 2 deletions doc/_admin-guide/150_Statistics_of_syslog-ng/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ description: >-

## Recommended: Structured, selective methods

- Using the syslog-ng-ctl
query command.
- Using the syslog-ng-ctl query command.

For further information about using syslog-ng-ctl commands, see
The {{ site.product.short_name }} manual pages.
Expand Down