OTTL doesn't support log body being anything other than a string

[Mrod1598]

Component(s)

No response

What happened?

Description

When using the filter processor and trying to point at a map key in body to execute a regex match, it doesn't work.

Steps to Reproduce

Attempt to filter on body that has a multiple keys

    processors: 
      - filter:
          logs:
            log_record:
              - 'IsMatch(body["status"] >= "2.*") == false'
              - 'body["status"] >= 200'

Expected Result

To be able to index body as a map like attributes

Solution

I believe the fix would be duplicating the logic for attributes for body here.

Collector version

v0.68.0

[frzifus]

cc @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

[github-actions]

Pinging code owners for processor/filter: @TylerHelmuth @boostchicken. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

@Mrod1598 OTTL doesn't support indexing on the body field so body["status"] and body["status"] will both access just the body field. (As an aside, that is not a great user experience. I'll update OTTL to error if you try to index a field that doesn't support indexing)

Also your syntax for IsMatch is not correct. IsMatch takes a string and checks it against a regex and returns true if it matches and false if it doesn't. It does not support inequalities as a parameter. In fact, OTTL's grammar only accepts inequalities in the where clause and no where else.

At the moment ottllogs returns the body as whatever pcommon.Value type is used, so if body is a string it will return string and if it is an int it will return int64, and so on. You can see that happening here and here.

If you do need to get access to a status field within body then take a look at using the transform processor before the filterprocessor to manipulate the data to a state that the filter processor can use. I think the parse_json function will be useful.

[djaglowski]

I think the parse_json function will be useful.

This suggests that the body is not already parsed, in which case I agree OTTL should treat it as a string (because it is a string).

However, the body can and often is actually structured (i.e. map[string]interface). In my opinion, OTTL should treat structured bodies the same as it treats attributes. Is there a technical reason OTTL shouldn't support this?

[TylerHelmuth]

@djaglowski we could add indexing for the body path, erroring if body is returned as anything but a pcommon.Map.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/filter: @TylerHelmuth @boostchicken

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

Related to #20754. Once we have complex indexing for paths and converters we could add the capability to body.