Skip to content

Commit

Permalink
Add NATS client as option for publishing container logs (#369)
Browse files Browse the repository at this point in the history 
* Add NATS client as option for publishing container logs

Signed-off-by: Waldemar Quevedo <waldemar.quevedo@gmail.com>
  • Loading branch information
Waldemar Quevedo authored and Misty Stanley-Jones committed Nov 22, 2016
1 parent 2b05451 commit 1c8370d
Show file tree
Hide file tree
Showing 3 changed files with 148 additions and 41 deletions.
3 changes: 2 additions & 1 deletion engine/admin/logging/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,5 @@ title: Logging drivers
* [Amazon CloudWatch Logs logging driver](awslogs.md)
* [Splunk logging driver](splunk.md)
* [ETW logging driver](etwlogs.md)
* [Google Cloud Logging driver](gcplogs.md)
* [Google Cloud Logging driver](gcplogs.md)
* [NATS Logging driver](nats.md)
88 changes: 88 additions & 0 deletions engine/admin/logging/nats.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
description: Describes how to use NATS for publishing log entries
keywords: NATS, nats.io, messaging, docker, logging, driver
title: NATS logging driver
---

Docker logging driver for sending container the logs as events published to NATS in JSON format.

## Usage

You can configure the default logging driver by passing the `--log-driver`
option to the Docker daemon:

```bash
$ dockerd --log-driver=nats
```

You can set the logging driver for a specific container by using the
`--log-driver` option to `docker run`:

```bash
$ docker run --log-driver=nats ...
```

This log driver does not implement a reader so it is incompatible with `docker logs`.

## nats options

You can use the `--log-opt NAME=VALUE` flag to customize the logging driver
for NATS:

| Option | Required | Description |
|-----------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------|
| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. |
| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. |
| `tag` | optional | Specify tag for message. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. |
| `nats-servers` | optional | NATS cluster nodes separated by commas. e.g. `nats://127.0.0.1:4222,nats://127.0.0.1:4223`. Defaults to `localhost:4222` |
| `nats-max-reconnect` | optional | Maximum attempts that the driver will try to connect before giving up. Defaults to infinite (`-1`) |
| `nats-subject` | optional | Specific subject to which logs will be published. Defaults to using `tag` if not specified |
| `nats-user` | optional | Specify user in case of authentication required |
| `nats-pass` | optional | Specify password in case of authentication required |
| `nats-token` | optional | Specify token in case of authentication required |
| `nats-tls-ca-cert` | optional | Specified the absolute path to the trust certificates signed by the CA |
| `nats-tls-cert` | optional | Specifies the absolute path to the TLS certificate file |
| `nats-tls-key` | optional | Specifies the absolute path to the TLS key file |
| `nats-tls-skip-verify` | optional | Specifies whether to skip verification by setting it to `true` |

Below is an example usage of the driver for sending logs to a node in a
NATS cluster to the `docker.logs` subject:

```bash
$ docker run --log-driver=nats \
--log-opt nats-subject=docker.logs \
--log-opt nats-servers=nats://nats-node-1:4222,nats://nats-node-2:4222,nats://nats-node-3:4222 \
your/application
```

By default, the tag is used as the subject for NATS, so it has to be a valid
subject in case subject it is left unspecified:

```bash
{% raw %}
$ docker run --log-driver nats \
--log-opt tag="docker.{{.ID}}.{{.ImageName}}"
your/application
{% endraw %}
```

Secure connection to NATS using TLS can be customized by setting `tls://` scheme
in the URI and absolute paths to the certs and key files:

```bash
docker run --log-driver nats \
--log-opt nats-tls-key=/srv/configs/certs/client-key.pem \
--log-opt nats-tls-cert=/srv/configs/certs/client-cert.pem \
--log-opt nats-tls-ca-cert=/srv/configs/certs/ca.pem \
--log-opt nats-servers="tls://127.0.0.1:4223,tls://127.0.0.1:4222" \
your/application
```

Skip verify is enabled by default, in order to deactivate we can specify `nats-tls-skip-verify`:

```bash
docker run --log-driver nats \
--log-opt nats-tls-skip-verify \
--log-opt nats-servers="tls://127.0.0.1:4223,tls://127.0.0.1:4222" \
your/application
```
98 changes: 58 additions & 40 deletions engine/admin/logging/overview.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,46 +25,44 @@ supported:
| `splunk` | Splunk logging driver for Docker. Writes log messages to `splunk` using HTTP Event Collector. |
| `etwlogs` | ETW logging driver for Docker on Windows. Writes log messages as ETW events. |
| `gcplogs` | Google Cloud Logging driver for Docker. Writes log messages to Google Cloud Logging. |
| `nats` | NATS logging driver for Docker. Publishes log entries to a NATS server. |

The `docker logs` command is available only for the `json-file` and `journald`
The `docker logs` command is only available for the `json-file` and `journald`
logging drivers.

The `labels` and `env` options add additional attributes for use with logging
drivers that accept them. Each option takes a comma-separated list of keys. If
there is collision between `label` and `env` keys, the value of the `env` takes
precedence.
there is a collision between `label` and `env` keys, the value of the `env`
takes precedence.

To use attributes, specify them when you start the Docker daemon. For example,
to manually start the daemon with the `json-file` driver, and include additional
attributes in the output, run the following command:
Specify attributes when you start the Docker daemon. For example, the following
command starts the daemon with the `json-file` driver and includes additional
attributes in the output.

```bash
$ dockerd \
--log-driver=json-file \
--log-opt labels=foo \
--log-opt env=foo,fizz
$ dockerd --log-driver=json-file \
--log-opt labels=foo \
--log-opt env=foo,fizz
```

Then, run a container and specify values for the `labels` or `env`. For
example, you might use this:
This command runs a container with specific values for the `labels` or `env`.

```bash
$ docker run -dit --label foo=bar -e fizz=buzz alpine sh
```

This adds additional fields to the log depending on the driver, e.g. for
`json-file` that looks like:
This adds additional fields to the log depending on the driver. If you use
`json-file`, it might add the attributes as follows:

```json
"attrs":{"fizz":"buzz","foo":"bar"}
```


## json-file options

The following logging options are supported for the `json-file` logging driver:

```no-highlight
```none
--log-opt max-size=[0-9]+[kmg]
--log-opt max-file=[0-9]+
--log-opt labels=label1,label2
Expand All @@ -87,7 +85,7 @@ from the newest log file.

The following logging options are supported for the `syslog` logging driver:

```no-highlight
```none
--log-opt syslog-address=[tcp|udp|tcp+tls]://host:port
--log-opt syslog-address=unix://path
--log-opt syslog-address=unixgram://path
Expand Down Expand Up @@ -205,7 +203,7 @@ By default, Docker uses the first 12 characters of the container ID to tag log
messages. Refer to the [log tag option documentation](log_tags.md) for
customizing the log tag format.

The `labels` and `env` options are supported by the gelf logging
The `labels` and `env` options are supported by the `gelf` logging
driver. It adds additional key on the `extra` fields, prefixed by an
underscore (`_`).
```json
Expand Down Expand Up @@ -240,24 +238,24 @@ For example, to specify both additional options:
```bash
{% raw %}
$ docker run -dit \
--log-driver=fluentd \
--log-opt fluentd-address=localhost:24224 \
--log-opt tag="docker.{{.Name}}" \
alpine sh
--log-driver=fluentd \
--log-opt fluentd-address=localhost:24224 \
--log-opt tag="docker.{{.Name}}" \
alpine sh
{% endraw %}
```

If container cannot connect to the Fluentd daemon on the specified address and
`fluentd-async-connect` is not enabled, the container stops immediately.
For detailed information on working with this logging driver,
see [the fluentd logging driver](fluentd.md)

If cthe ontainer cannot connect to the Fluentd daemon on the specified address
and `fluentd-async-connect` is not enabled, the container stops immediately. For
detailed information about working with this logging driver, see [the fluentd
logging driver](fluentd.md)

## Amazon CloudWatch Logs options

The Amazon CloudWatch Logs logging driver supports the following options:

```bash

```none
{% raw %}
--log-opt awslogs-region=<aws_region>
--log-opt awslogs-group=<log_group_name>
Expand All @@ -266,41 +264,61 @@ The Amazon CloudWatch Logs logging driver supports the following options:
{% endraw %}
```

For detailed information on working with this logging driver, see [the awslogs
logging driver](awslogs.md) reference documentation.
For detailed information about working with this logging driver, see
[the awslogs logging driver](awslogs.md) reference documentation.

## Splunk options

The Splunk logging driver requires the following options:
The `splunk`` logging driver requires the following options:

```no-highlight
```none
--log-opt splunk-token=<splunk_http_event_collector_token>
--log-opt splunk-url=https://your_splunk_instance:8088
```

For detailed information about working with this logging driver, see the
For detailed information about working with the `splunk` logging driver, see the
[Splunk logging driver](splunk.md) reference documentation.

## ETW logging driver options

The etwlogs logging driver does not require any options to be specified. This
logging driver forwards each log message as an ETW event. An ETW listener
can then be created to listen for these events.
The `etwlogs` logging driver forwards each log message as an ETW event. An ETW
listener can then be created to listen for these events. This driver does not
accept any options.

The ETW logging driver is only available on Windows. For detailed information
on working with this logging driver, see [the ETW logging driver](etwlogs.md)
about working with this logging driver, see [the ETW logging driver](etwlogs.md)
reference documentation.

## Google Cloud Logging options

The Google Cloud Logging driver supports the following options:

```no-highlight
```none
--log-opt gcp-project=<gcp_projext>
--log-opt labels=<label1>,<label2>
--log-opt env=<envvar1>,<envvar2>
--log-opt log-cmd=true
```

For detailed information about working with this logging driver, see the
[Google Cloud Logging driver](gcplogs.md). reference documentation.
For detailed information about working with the Google Cloud logging driver, see
the [Google Cloud Logging driver](gcplogs.md). reference documentation.

## NATS logging options

The NATS logging driver supports the following options:

```none
--log-opt labels=<label1>,<label2>
--log-opt env=<envvar1>,<envvar2>
--log-opt tag=<tag>
--log-opt nats-servers="<comma separated list of nats servers uris>"
--log-opt nats-max-reconnect="<max attempts to connect to a server>"
--log-opt nats-subject="<subject where logs are sent>"
--log-opt nats-tls-ca-cert="<absolute path to cert>"
--log-opt nats-tls-cert="<absolute path to cert>"
--log-opt nats-tls-key="<absolute path to cert>"
--log-opt nats-tls-skip-verify="<value>"
```

For detailed information, see [the NATS logging driver](nats.md) reference
documentation.

0 comments on commit 1c8370d

Please sign in to comment.