MasterService#patchVersions is rather inefficient

[DaveCTurner]

Today MasterService#patchVersions creates a new Metadata$Builder, increments the version, and then calls build() which expensively recomputes a bunch of data structures, e.g. Metadata#indicesLookup. This is quite inefficient, we could just copy those data structures from the previous Metadata and save a bunch of work and garbage-generation.

This is what it looks like in a flamegraph:

That's only ~7.5% of the total time on this otherwise busy master, but we could still cut this in half.

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[original-brownbear]

Link #77466

This one is still very interesting to the many shards effort. I just looked at a recent profile and as we make improvements to various CS update steps this is actually starting to make up a larger and larger portion of what the master update thread spends its time on.
We can probably do a similar thing to what the fix here looks like to org.elasticsearch.cluster.routing.allocation.IndexMetadataUpdater#applyChanges as well, which also gets hit by rebuilding the full index lookup even though it only changes part of the Metadata that have not relevance to the index lookup as well.

[original-brownbear]

Looks like #79004 is going to mostly address this now from the looks of it (I just benchmarked it and that PR gets us a large chunk of the obvious improvements that are possible for this method). Please hold off on working on this for now.