Support log json log format

[nicosto]

Feature Request

Opening a feature request kicks off a discussion.

Proposal:

enable json output for log

Current behavior:

ubuntu@ubuntu-lts:~$ kubectl -n intam logs telegraf-f494bc567-zmgq9 |head -50
2020-12-04T04:02:22Z I! Starting Telegraf
2020-12-04T04:02:22Z I! Using config file: /etc/telegraf/telegraf.conf
2020-12-04T04:02:22Z I! Loaded inputs: cpu internal internal kafka_consumer
2020-12-04T04:02:22Z I! Loaded aggregators:
2020-12-04T04:02:22Z I! Loaded processors: enum
2020-12-04T04:02:22Z I! Loaded outputs: prometheus_client kafka kairosdb health
2020-12-04T04:02:22Z I! Tags enabled:
2020-12-04T04:02:22Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:10s
2020-12-04T04:02:22Z D! [agent] Initializing plugins
2020-12-04T04:02:22Z D! [agent] Connecting outputs
....

Desired behavior:

in a Kubernetes infrastructure to be able to enable log JSON format support to handle multiline output.

Use case:

log can be across multi-line especially in debug mode. When stored in a backend (e.g. elasticsearch) each line creates an entry.

Thanks
Nicolas

[sjwang90]

@nicosto Can you provide an example of the log JSON data you'd like to read in?

[idrennanvmware]

+1 for this request. We use structured logging whenever possible across our log aggregators and most products already support this (json format or flat).

[serut]

From :

2020-12-04T04:02:22Z I! Starting Telegraf

To :

{"level":"info","msg":"Starting Telegraf","t":"2020-12-04T04:02:22.014980542Z"}

[srebhan]

@nicosto, @idrennanvmware and @serut please test the binary in PR #15751, available once CI finished the tests, and let me know if this fixes this issue! You need to set logformat = "structured" in the agent section to enable structured logging...