Allow templating headers in `http` sink

[binarylogic]

For certain services the ability to dynamically set HTTP headers based on log content is important. From a user on a related project:

I am sending Docker container logs to Sumo Logic, and would like to dynamically set log metadata based on the incoming log records. Sumo Logic's HTTP receivers accept custom source names, hosts, and categories by setting HTTP headers named X-Sumo-Name, X-Sumo-Host, X-Sumo-Category.

Describe the solution you'd like

I would like a way to set dynamically header values in the http output plugin. For example, let's say there is a record key named category. There should be a way to use this as the header value.

[1154103724, {"key1"=>12345, "category"=>"abc"}]

[sinks.sumologic]
    Name           http
    Match          *
    Host           127.0.0.1
    Port           9000
    Header         X-Key-A Value_A
    Header_record  X-Key-B category
    URI            /something

In the above configuration, the header X-Key-A would have the value Value_A, but X-Key-B would have the value abc from the record.

Additional context

Without the ability to customize the log metadata via headers, I cannot effectively sort and manage logs in Sumo Logic.

[binarylogic]

Thinking about this more, how do we handle multiple different values? Does it create multiple buffers / requests? In a way, I'm thinking this should be addressed with a higher-level feature like a "group_by" transform.

[jalaziz]

Still new to Rust, but could this be solved with the relatively new PartitionHttpSink?

[binarylogic]

cc @bruceg on that one.

[bruceg]

Yes, PartitionHttpSink looks to be the ideal target for this kind of modification, and would end up being ideal sample code for other HTTP sinks to adapt when they need this kind of enhancement. The key to the partition types would be the HTTP headers so that the sink batches events with like headers together.

[jszwedko]

Closed by #7189

[jalaziz]

@jszwedko Has this actually been fixed? #7189 was an issue that was closed in favor of this one. I can't seem to find a PR related to either.

[jszwedko]

Hi @jalaziz !

Apologies, you are right, I misread this as just allowing headers to be set, but I see this is about allowing templated HTTP headers. I'll reopen.