Honor data_stream.dataset in input packages

[jsoriano]

Input packages need to set data_stream.dataset in the agent configuration to actually send data to the configured dataset.

The configured dataset is set by the Fleet UI through a data_stream.dataset variable for input packages.

Following this, assume that system test configurations can include a data_stream.dataset variable, and use it in the places where a dataset is expected:

• In the data stream name.
• In the package component template.
• In the package policy.
• In the list of expected datasets.

Fixes #1806

[jsoriano]

This seems to be required in input packages to actually leverage this option. I think we should handle this in a different way. Ideally Fleet should add this if this is required in all input packages. cc @kpollich @mrodm

[kpollich]

Yes I think you are right. Fleet is enforcing this as a requirement, so Fleet should add this to the input package if it isn't explicitly set as a "default". Dynamically altering the handlebars template before rendering it isn't something we've done before in Fleet, but I don't think it will be more than some string manipulation. There's potential for complexity, though, to be sure. I created elastic/kibana#184008 to capture this on the Fleet side.

[jsoriano]

Thanks 👍

I think this config could be appended if not already set.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 56820e3

Failed CI Steps

• :go: Integration test: aws (independent agent)

History

cc @jsoriano