Skip to content

Filebeat docker input: support reading of labels and env vars written by docker JSON File logging driver #8358

Closed
@passing

Description

@passing

The docker input of filebeat is specialized to read log files written by the docker JSON File logging driver:

In the logging configuration of docker, it is possible to define which labels and environment variables should be included in the json logs:

{
    "log-driver": "json-file",
    "log-opts": {
        "labels": "stage",
        "env": "APPLICATION_NAME,MARATHON_APP_ID,MESOS_TASK_ID"
    }
}

in the logfiles, the labels and environment variables are added under the key "attrs" then:

{
  "log": "...log message...\n",
  "stream": "stdout",
  "attrs": {
    "APPLICATION_NAME": "test-service",
    "MARATHON_APP_ID": "/test/test-service",
    "MESOS_TASK_ID": "test_test-service.f1688116-b502-11e8-b3f9-02424fbc88a4",
    "stage": "prod"
  },
  "time": "2018-09-19T14:41:15.191384718Z"
}

with filebeat version 6.4, only "log", "stream" and "time" are used from the json log-file, "attrs" are not included.

(Using the "add_docker_metadata" processor only allows to add labels but no environment variables to events)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions