[exporter/elasticsearch] When using mapping.mode: ecs, drop scalar fields in case of object/scalar conflicts

[ycombinator]

Component(s)

exporter/elasticsearch

Is your feature request related to a problem? Please describe.

When mapping.mode: ecs is used, certain OTel + SemConv fields are converted to ECS fields to ensure that dashboards and other assets currently using ECS fields continue to work. Due to this conversion, it's possible to run into object/scalar conflicts in the source field set, for example, {"foo": 1, "foo.bar": 2}.

Describe the solution you'd like

In such cases, we should drop the scalar value. For example, {"foo": 1, "foo.bar": 2} should be converted to {"foo": { "bar": 2}}.

Describe alternatives you've considered

No response

Additional context

No response

[ycombinator]

cc: @axw

[github-actions]

Pinging code owners for exporter/elasticsearch: @JaredTan95 @ycombinator @carsonip. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

This was filed by a code owner, so I'm removing needs triage.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/elasticsearch: @JaredTan95 @ycombinator @carsonip

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

This is still relevant. I think it's only relevant with the config dedup: false, but need to confirm that. I'll take a stab at it as part of #33776

[axw]

It's only relevant in the case of dedup: false, which I'm removing in #33776. I've added a test case to that PR.

The existing behaviour is that {"foo": 1, "foo.bar": 2} will be translated to {"foo": {"value": 1, "bar": 2}} with de-dotting. If the input is rather {"foo": 1, "foo.value": 2}, then the result will be {"foo": {"value": 2}}.

I'm not a fan of this magical ".value" addition, but inclined to leave it alone for now.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/elasticsearch: @JaredTan95 @ycombinator @carsonip

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.