Incorrect output of the "tracing" fields in the Beats yml file

[webmat]

Just like the base fields, the tracing fields are not nested under the name of the field set. So it's not base.@timestamp, it's @timestamp, and it's not tracing.trace.id, it's trace.id.

In the Beats field yaml file the ECS project generates, the tracing fields are incorrectly nested under a tracing section, which means Beats interprets the field names incorrectly (tracing.trace.id).

This is a bug, these fields shouldn't be nested this way.

In order to fix this issue, we should remove this nesting in the Beats yml output. Just like @timestamp and other base fields are not nested under a field group.

I think this bug fix will be at minimum backported to 1.7. Thoughts welcome on this, is there a need to backport to 1.6 as well?

The Beats PR elastic/beats#22571 to import ECS 1.7 should be adjusted with these changes, once the bug fix is ready. cc @andrewstucki

[webmat]

@felixbarny I think this one needs to be on your radar as well.

@thomheymann Thanks for helping us find this issue 👍

[felixbarny]

What's the impact of this? I suppose the index pattern would not be correct so that trace.id will be dynamically mapped to a text type instead of keyword, allocating a bit more space, right? Any other impacts?

[webmat]

Both the Beats dynamic templates and the one in the ECS sample Elasticsearch template matched the "string" fields as keyword, overriding the default Elasticsearch convention. So users populating any of the tracing fields via Beats or a solution based on the ECS sample template would already have them mapped as keyword.

So there should be no impact on any solutions that use these fields.