[APM] 8.0 breaking changes

[sorenlouv]

TODO: Make list of breaking changes in apm ui, and whether they should be migrated via the upgrade assistant.

Decided

• Deprecate and remove unused xpack.apm.* config settings ([APM] Deprecate obsolete xpack.apm.serviceMapEnabled config options #96278)
• Deprecate apm_oss app ([APM] Deprecate apm_oss app #108160)
• Prepare for unsampled transactions being dropped ([APM] Support for dropping unsampled transactions (meta) #108466)
  • This will cause incorrect doc_count (must be multiplied by sampling rate)
  • tail-based sampling will cause metrics like avg to be incorrect
  • Alerting, Correlations and ML jobs still use transaction documents
  • Observability Overview doesn't show a callout when falling back to transactions
• Drop apm_user ([APM] Replace apm_user with stack-wide roles (editor / viewer) #116760)

Up for discussion
These can be tabled for now:

• Use/require aggregate_metric_double for span/transaction breakdown metrics: Store various metrics (sum,count,etc) as aggregate_metric_double apm-server#4612
• consider creating a single identifier for a service (combination of service.name, service.environment, tranasction.type)
• consider making service.environment required in agent (Set default service.environment  apm-server#4468)
• Consider removing the ALL option in agent configuration

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[dgieselaar]

Some other fields we should consider making required:

• span.subtype (used in breakdown data)
• transaction.result (used in transaction result charts)
• service.node.name (used for system metrics)

Simply put, we should look into making all fields that we aggregate on via a terms or composite aggregation required, as using missing or missing_bucket means that we miss out on specific ES optimisations.

[sorenlouv]

Thanks @dgieselaar. I'll talk to the server team about this.