Move runtime fields stats to server

[javanna]

Runtime fields usage is currently reported as part of the xpack feature usage API. Now that runtime fields are part of server, their corresponding stats can be moved to be part of the ordinary mapping stats exposed by the cluster stats API.

This is the last bit of functionality left in the xpack runtime-fields plugin, hence we can remove the plugin as part of this change too.

This PR removes the runtime-fields feature from the xpack/usage and xpack/info output, in favour of reporting runtime_field_types stats as part of the output of the cluster stats API together with the existing field types stats.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[jtibshirani]

I took a look at the approach, and the approach to BWC makes sense to me. Two high-level comments:

• This is technically a breaking API change, so I wonder if we should only make it only master. Or do our telemetry APIs have special status, and we're more lenient with breaks?
• It'd be great to maintain some YML tests that exercise the xpack usage API in mixed-cluster scenarios. This would give us extra confidence that the xpack usage API behaves as expected when runtime fields are enabled in a mixed 7.12 and 7.13 scenario.

[javanna]

This is technically a breaking API change, so I wonder if we should only make it only master. Or do our telemetry APIs have special status, and we're more lenient with breaks?

Agreed, I worked under the assumption that xpack/usage is an internal API and that moving the telemetry stats to the right place as soon as possible is ideal given that telemetry is the only thing left on the xpack runtime fields plugin. These fields are not even mapped on the telemetry cluster so we should be safe here. I could leave the runtime fields feature in the response output in 7.x, but with the empty runtime_field_types array: that would avoid breaking the response format, but I am not sure it would bring any value.

It'd be great to maintain some YML tests that exercise the xpack usage API in mixed-cluster scenarios. This would give us extra confidence that the xpack usage API behaves as expected when runtime fields are enabled in a mixed 7.12 and 7.13 scenario.

We do have some which indeed failed and led me to keeping the named writeable around in 7.x. As for the expected behaviour in a mixed cluster: I think that we should ensure things don't break, but there is no guarantee that runtime fields stats will be accurate. Mixed clusters are a temporary scenario in which runtime field stats will be inaccurate depending on which node is hit with the request, until the cluster is all upgraded to 7.13+ nodes.

I was chatting to @rjernst about this the other day and I would be keen to have his opinion too.

I am thinking that getting in the 7.x first (#69542), and then this one should make sure that there are no problems with bwc tests.

[jtibshirani]

We do have some which indeed failed and led me to keeping the named writeable around in 7.x

I was thinking of a test where runtime fields were actually present in the mapping, which I don't think we test anymore? This is just a small suggestion though.

[rjernst]

LGTM

[rjernst]

This is technically a breaking API change

I don't see it as breaking, since the xpack API this was a part of was not meant for external consumption. It's entire purpose is telemetry, and we don't document the API.