[Unified Waterfall] Add support for coloring OTel spans with the 'kind' field

[iblancof]

Summary

Closes #228239

This PR improves the waterfall UI for non-processed OTel data in traces that belong to a single service by adding support for the kind field to determine item coloring. When there are multiple services, the legend and coloring continue to be based on service.name.

For APM / processed OTel, if an item in the waterfall doesn’t have a span.type or span.subtype, we normally use the service.name in the legend, because transactions can miss these fields. However, for non-processed OTel tracing, every span always includes a kind field, so there’s no need to fall back to service.name in this case.

Before	After

How to test

• Ingest non-processed OTel data (e.g., using https://github.com/smith/elastic-stack-docker-compose)
• Setup an Observability solution space
• Run a query to get traces with only one service and spans with different kinds

FROM traces* | STATS 
    service_count = count_distinct(service.name), 
    kind_count = count_distinct(kind)
  BY trace.id | WHERE service_count ==  1 AND kind_count > 1

• Copy the trace.id of any resulting trace and run another query to fetch any document with that trace.id
• Open document flyout
• Open full screen waterfall from the Overview tab
• The trace should display a different color for each kind present in the trace

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[cauemarcondes]

I was wondering if it would be possible to not have these two fields in the unified waterfall interface. By having both fields the waterfall now "knows" between otel and apm documents differences.

Do you think we could join these two fields in one? Maybe keeping only kind becasue it's an otel field. We'd then color the waterfal based on the service.name or the kind (that could be both spanType (APM docs) or kind (Otel docs)).

[smith]

These fields mean two different things. There is a proposal to add something more like span.type to OpenTelemetry.

[iblancof]

Thanks for raising this!

I was thinking about this before starting the work, I even considered just adding kind to the spanType to avoid making too many changes. But then I thought OTel users would probably prefer to see the data shown and named the way it really is.

That said, you’re right, the Waterfall shouldn’t really care where the data comes from. It should just build the UI based on the data it gets.

When we worked on the span failure/error display, we first handled it differently depending on the data source, but in the end we made it generic enough so the Waterfall didn’t need to know about that, it all came down to how we passed the data in.

I’ll be doing some changes to keep the Waterfall simple in the same way.

The only thing I’d like to keep is showing “Kind” or “Type” as the legend title, depending on what we’re showing. Using the same label for different things across APM and OTel could be confusing.

[iblancof]

@cauemarcondes details about the changed approach explained here.

[iblancof]

After this comment, I decided to go with a simpler approach and just use a type property in the TraceItem to match the legend with the row/item coloring.

To ensure the APM behavior hasn’t been negatively impacted, here are a few different use cases:

Data source	Single service	Multi service
APM/processed OTEL
Non-processed OTel

Regarding the legend title, I decided to keep just the "Type" label to avoid overcomplicating the logic. That said, if anyone strongly disagrees, I'm happy to revisit the solution and explore a dynamic alternative. However, that would likely require introducing a different kind of configuration for the legend, rather than relying on this new single type property.

Changes done in acf82c4

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: effe744

Failed CI Steps

• Check Types

History

• 💔 Build #327896 failed 2cbd8d5
• 💔 Build #327881 failed 18e411d
• 💔 Build #327141 failed b3b70cb

cc @iblancof

[cauemarcondes]

LGTM