Implement filtering mechanism in the Logging service

[mshustov]

Blocker for #13241
Once we move request/response logging to the new platform, we need to provide a way to censor sensitive data in the logs (e.g. authorization or cookie headers).
Evaluate how much work it would be to support a filtering mechanism compatible with elasticsearch logging settings https://logging.apache.org/log4j/2.x/manual/filters.html

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[pgayvallet]

• In l4j, that kind of filter is usually based on informations present in the MDC. We would not be able to rely on such mechanism. Do we have anything better than applying regexp for that kind of filter then?

• Do we want to filter (remove) the whole log message when it contains sensitive data, or do we just want to remove/obstruct the sensitive data in the log message but still log it.

[joshdover]

Do we want to filter (remove) the whole log message when it contains sensitive data, or do we just want to remove/obstruct the sensitive data in the log message but still log it.

I strongly prefer we just filter the sensitive data out but still log the message.

[mshustov]

In l4j, that kind of filter is usually based on informations present in the MDC. We would not be able to relies on such mechanism. Do we have anything better than applying regexp for that kind of filter then?

LP relies on Hapi log output for request/response and filters data in JSON before formatting them. We will control the output format for request/response logging when #13241 lands. We can log additional data as MetaData and apply the filter to metadata in JSON format.

Do we want to filter (remove) the whole log message when it contains sensitive data, or do we just want to remove/obstruct the sensitive data in the log message but still log it.

I strongly prefer we just filter the sensitive data out but still log the message.

The current implementation allows only 2 operations:

• remove a property
• censor a string property (the full string or via regexp)

Example https://github.com/elastic/kibana/blob/8e9a8a84dccfa7965ce8a22362885e6cdef8b51f/src/legacy/server/logging/apply_filters_to_keys.js