[pkg/ottl] Add the ottl.ParserCollection utility

[edmocosta]

Description

This PR is part of #29017, and adds the parser collection utility into the ottl package.

This utility is based on the existing transformprocessor's ParserCollection concept and is used to group context's parsers, abstracting the logic to infer the context from the statements, and/or rewriting them adding their paths's context if necessary.

In summary, the general idea is: given the configured contexts, parsers and statements converters, pick the right parser, and transform the parsed result (ottl.Statements[K]) into something common to the parser collection ([R]). For more details, please have a look at the transformprocessor` refactoring on the draft using this utility.

Given the ottl.Parser[K]'s have different type restrictions/generics (e.g. TransformContext), this utility heavily rely on the Go's reflection api, being the exposed interface type-safe, but manipulating untyped objects under the hood.

Considering this code is only executed during the bootstrap, IMO, it should be fine using reflection. I couldn't think about any other solution that would make this utility as generic as it's without reflection. Another approach would be hard-code all the existing ottl contexts's parsers on the collection, having fixed WithParserX functions per context and generating a direct dependency on the ottl/contexts package.

Link to tracking issue

#29017

Testing

Unit tests

Documentation

No changes

[TylerHelmuth]

Do we need this?

[edmocosta]

Same as #36174 (comment), but in this case it's not really necessary considering this interface is declared inside this same file.
We can definitely replace it by a comment, but IMO, I'd keep it as a safe-guard, considering changing this function signature could break the parser collection here, and in all its external usages. WDYT?

[TylerHelmuth]

Is this enforcing that we'll be able to make the reflective call to ParseStatements?

[edmocosta]

Yes, it's an interface safe guard and is only being used to statically check the ParseStatements signature at compile time. It fails to compile If for some reason the ottl.Parser.ParseStatements changes (which could make the reflective call to fail).

[TylerHelmuth]

Does this type need a generic? I don't see where it uses it.

[edmocosta]

Good catch! that's correct, it does not need to be generic.

[TylerHelmuth]

Is S StatementsGetter required? Would specifying parameters of type StatementGetter sufficient? Or even better can we get away with []string?

[edmocosta]

Is S StatementsGetter required?

Not really, I think we can drop it from the ParserCollection. It's only necessary for the ParsedStatementConverter.

[TylerHelmuth]

Instead of taking an interface can this take []string like the parser's ParseStatements?

[edmocosta]

That'd possible, but the idea of making it an interface was giving theParsedStatementConverter functions more flexibility to receive extra data, without the need of adding an extra argument for some custom data.

For example, the transformprocessor builds the parser collection and them invokes the ParseContextStatements function passing on a custom type ContextStatements argument. Besides the Statements []string property, that type has a Conditions []string property used to convert (see) the parsed ottl.Statement[K] into [R], which is only known after the parser collection is built.

In summary, having it as an interface allows the ParsedStatementConverter functions to access customised ParseStatements input arguments, avoiding extra parameters on the ParsedStatementConverter and making the parser collection reusable.

WDYT?

[TylerHelmuth]

Instead of taking an generic S can this take []string like the parser's ParseStatements?

[TylerHelmuth]

I'm not against the use of reflection and generics (OTTL uses both a lot already), but we've also taken steps in the ottl context packages to make it easier to use OTTL in a component without needing to know about OTTL's generics. ottllog.NewParser exists so users don't have to do ottl.Parser[ottllog.TransformContext]. I believe we'll want to do something similar with this new API - It needs to be generic enough for others to provide their own context collections (thanks reflection), but our contexts should be able to hide this feature.

I believe this solution supports this but I want to call it out. In my head I want to still be able to do ottllog.NewParser and it takes care of using a ParserCollection behind the scenes, enforcing the valid contexts.

[edmocosta]

I believe this solution supports this but I want to call it out. In my head I want to still be able to do ottllog.NewParser and it takes care of using a ParserCollection behind the scenes, enforcing the valid contexts.

That's interesting, but I think it would require another type of API (builder perhaps) considering a single OTTL context (ottllog.NewParser) woudn't be able to build the whole parser collection. For example, if I want to parse OTTL log and metric contexts statements, I'd create a parser collection as:

logParser, _ := ottllog.NewParser(ottlfuncs.StandardFuncs[ottllog.TransformContext](), telemetrySettings, ottllog.WithPathContextNames())
metricParser, _ := ottlmetric.NewParser(ottlfuncs.StandardFuncs[ottlmetric.TransformContext](), telemetrySettings, ottlmetric.WithPathContextNames())

pc, err := ottl.NewParserCollection[any](
  telemetrySettings,
  ottl.WithParserCollectionContext("log", &logParser, {...}),
  ottl.WithParserCollectionContext("metric", &metricsParser, {...}),
)

pc.ParseStatements(statements{[]string{`set(log.attributes["key"], "value")`}})

It indeed does not completely hides the generic type, but I think it's somehow similar to the NewParser functions.

In my head I want to still be able to do ottllog.NewParser and it takes care of using a ParserCollection behind the scenes, enforcing the valid contexts.

The valid contexts are still being enforced by the underline parser, and not by the parser collection, for example, given the previous parser collection and the following log (new flat structure) statements:

- set(log.attributes["key"], "value")  # ok
- set(log.attributes["key"], attributes["value"]) # Parser fails and report a missing context for the `attributes["value"]` path
- set(log.attributes["key"], log.attributes["value"]) # ok
- set(log.attributes["key"], log.resource.attributes["value"]) # ok
- set(log.attributes["key"], log.instrumentation_scope.attributes["value"]) # ok
- set(log.attributes["key"], log.metric.attributes["value"]) # Parser fails as `metric` is not a valid path context name for the ottllog.NewParser (WithPathContextNames)
- set(trace.attributes["key"], trace.attributes["value"]) # ParserCollection fails as it does not support parsing `trace` context.

Please note that in this approach/initial version, the access to lower contexts would still be handled by the contexts themselves, for example logs, being the parser collection used only to parse top level contexts (inferred from the statements).