[OUTPUT] [DATADOG] Use Variables for dd_service property

[jmtt89]

Is your feature request related to a problem? Please describe.

It's very annoying need create a different match target for every dd_service, a much nicer use some record value, exactly like you do with dd_message_key

Describe the solution you'd like

Just allow to use variables from the record, just add a dd_service_key for define dd_service dynamically based on record information.

[jszwedko]

It is actually already possible to dynamically set the service field through the pipeline processors in Datadog. See https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=service#service-attribute . By default, it will look for either service or syslog.appname to determine the service, but you can configure it to look at additional attributes.

dd_service is useful for setting a hardcoded service name if the input lacks one, but if you don't set it you can configure Datadog to look in several places to find the service field. I'll open a PR to clarify the docs.

For example, given this configuration:

[INPUT]
  Name   dummy
  Dummy {"service": "foo", "message": "custom dummy"}

[OUTPUT]
  Name        datadog
  Match       *
  Host        http-intake.logs.datadoghq.com
  Port        443
  TLS         on
  compress    gzip
  apikey      XXXX

The service field from the incoming event will end up being used as the service tag in Datadog. Does that satisfy your use-case @jmtt89 ?

[jszwedko]

message can be handled in the same way so I don't think the dd_message_key is really necessary. It's possible its addition predated the functionality in Datadog to lookup the attribute though.

[jmtt89]

Hi @jszwedko , thanks for clarifying!!!

yes, configuring on datadog system solves the use case, I think, however, that it would be nice have a way to define it in the sender instead of depending on the platform to parse it.

In DataDog Pipeline -> "Preprocessing for JSON logs" is posible define these fields:

• date
• host
• service
• status
• traceId
• message

but some others like source, version of tags is not possible, maybe need create a custom pipeline or something like that, but I think it would be much easier to be able to define it in the output as with other fluent-bit outputs.