[pkg/ottl] Converter: MarshalKeyValue

[kuiperda]

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

Problem: Sending Windows Event Logs to QRadar

The recommended approach is to turn the event into key value pairs to put into the syslog rfc3164 message that QRadar expects and will automatically parse.

The current solution for this involves a long list of set and concat funcs to manipulate the parsed record into Key Value format.

Describe the solution you'd like

A simple MarshalKeyValue OTTL converter would take a pcommon.Map and convert it to a string of key value pairs with format k=v k2=v2...

It would be an inverse to the existing ParseKeyValue func. For detailed examples of behavior, see ParseKeyValue tests.

Note: There is some loss in the Parse func, i.e. trimming whitespace, removing wrapping ' and ", and escaping ".
Marshal would persist this loss so composing the funcs would not be an exact inversion.

I would advocate for customizable delimiters with default being = and (space).

Marshal would handle collisions with delimiters in the keys and values by wrapping the k/v in " after escaping any unescaped "

For nested maps and arrays, this simple case could automatically flatten them to the top level. But we may need to decide on a representation for nested maps and arrays and potentially allow customization of those symbols and delimiters as well. The other key consideration here is that Parse does not expect its input to be nested.

Example

MarshalKeyValue(target, Optional[delimiter], Optional[pair_delimiter])

Simple example usage (Initially added as a processor in the Bindplane Agent)

Describe alternatives you've considered

Adding a processor that handles Key Value marshaling. An initial approach to this has already been added in the Bindplane Agent.

Additional context

No response

[github-actions]

Pinging code owners:

• pkg/ottl: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

I think key/value format is simple enough that providing a marshaler makes sense. Handling of nested elements can just be a documented caveat that they are rendered as strings. Users can always use the dedicated Flatten function before marshaling if they like that output better.

[odubajDT]

Hello, I would like to look at this issue!

[kuiperda]

@odubajDT I am working on an implementation for this already. This issue is intended to gain consensus around a standard.

[kuiperda]

Initial implementation added in #35409

[crobert-1]

Removing needs triage based on response from project maintainer.

[kuiperda]

Completed with #35409