New values for `span.type` and introduce `span.subtype`, `span.action`

[roncohen]

At the moment our span types are in the form of span.type: db.mysql.query and indexed as a keyword.

They are loosely defined and you cannot easily search for example mysql. It hampers our ability to aggregate information across spans and makes the UI and data model more complicated than needed.

To fix this , we're going to introduce two new fields: span.subtype and span.action. We're also changing span.type from the dot-delimited format: span.type: db.mysql.query to just be the overarching span type: db, template, external:

Here's some examples:

| Field | Optional |
Description | Examples |
| --- | --- | --- | --- |
| type | No | Overarching type of span | * db
* cache
* template
* external |
| subtype | Yes | Sub division per type | * postgresql, mysql (db)
* jinja2, mustache (template)
* http , thrift (external) |
| action | Yes | Detail on what actually transpired | * query
* connect |

The values will not be enforced in the APM Server, but we should align on what the fields mean and some common values. Agents should be able to set the values for new instrumentations as they see fit.

Agent devs should make the actual alignment in this doc: https://docs.google.com/spreadsheets/d/1SmWeX5AeqUcayrArUauS_CxGgsjwRgMYH4ZY8yQsMhQ/edit#gid=644582948

OpenTracing

OpenTracing doesn't have a specification for these, however, we should make a mapping that allows us the set the span.type from the tags that users set using the OpenTracing API:

OT states:

Database type. For any SQL database, "sql". For others, the lower-case database category, e.g. "cassandra", "hbase", or "redis".

Here's a quick attempt at such a mapping:

OT tags	ElasticAPM types
db.type=sql	type=db, subtype=sql
db.type=cassandra	type=db, subtype=cassandra
db.type=redis	type=db, subtype=redis
http.* is set	type=ext, subtype=http
message_bus.* is set, peer.service=kafka	type=message_bus, subtype=kafka

As you can see, SQL is a bit of a special case in OpenTracing. We might be able to deduct the SQL software from peer.service, but it's also fine to start as type=db/subtype=sql for the OT case for now.

You may ask, what happens if both db.type=sql and message_bus.something is set:

1. I can't come up with a situation where that makes sense.
2. We'd need to just pick one of them to use in the mapping

Backwards compatibility

The UI could handle it gracefully and just show the type as type.split('.')[0], but anyone who relies on the types for dashboards etc. would be out of luck. We're OK with this breaking change in a minor, as I think it's not likely that anyone has built custom dashboards that rely on the type.

---

@elastic/apm-agent-devs @elastic/apm-ui please create an issue in your repo and add the link here / check the checkbox

• UI - [APM] Add span.subtype and span.action to the flyout kibana#26247
• Go Agent - Add span subtype/action apm-agent-go#323
• JS Agent - Support new values for span.type and introduce span.subtype, span.action apm-agent-rum-js#102
• Node.js - Split span type into 3 fields: Type, Subtype and Action apm-agent-nodejs#713
• Python implement span subtypes and actions apm-agent-python#377
• Ruby - New values for span.type and introduce span.subtype, span.action apm-agent-ruby#245
• Java - Add span.subtype and span.action apm-agent-java#324
• .NET - Add span.subtype and span.action apm-agent-dotnet#24
• APM Server Add span.subtype and span.action apm-server#1617

[axw]

message_bus.* is set | type=message_bus, subtype=kafka

Kafka is not implied here, so are we assuming that? What about RabbitMQ? We might be able to infer it from the component or service tags, but they're free form.

[roncohen]

good point. Looks like we might be able to gleam it from peer.service: https://github.com/opentracing-contrib/java-kafka-client/blob/51aa0a4f492bb715033be9c062fabacdaeb256d1/opentracing-kafka-client/src/main/java/io/opentracing/contrib/kafka/SpanDecorator.java#L75

[roncohen]

we might be able to use peer.service in more cases. I'll update the description

[felixbarny]

The value of peer.service is highly implementation specific, however.

[vigneshshanmugam]

Had a conversation with Ron in the office, Should we want to keep the API of span simple and continue with the dot notation or add a new argument to the span with subtype and action since our API deviates from OT already.

const span = new Span(name, type, subtype, action) 

[watson]

The API used by the user would be simpler and easier to use IMO if they just had to deal with a single string. We could then make it up to the APM Server to parse that single string into 3 parts. It could also be done by the agent internally of course, but I'd prefer if it was done by the APM Server

[roncohen]

My feeling is that the dot-notation is weird relic from a distant past. If we were to design this API from scratch today, I don't think we'd consider asking the user to concatenate three properties with a . and pass it as an argument.

[watson]

Yeah, I might be colored by our old ways of doing things. Though it's not uncommon to use strings with separators - e.g. OpenTracing tags uses dots as logical separators even though it's one big string. Anyways, I just felt it would be easier to write for the user, but maybe you're right 🤷‍♂️