Open
Description
Today IndexRoutingTable$Builder#build
constructs an all-new IndexShardRoutingTable
for each shard on every change. High-traffic clusters may have large numbers of shards per index, and today's behaviour yields many duplicate IndexShardRoutingTable
instances which together may consume a substantial amount of heap on the elected master.
Moreover IndexRoutingTable
is a SimpleDiffable
so we must transmit the whole table over the wire on any change, which prevents sharing on the receiving node and therefore yields many duplicate IndexShardRoutingTable
instances on the other nodes in the cluster.
We should:
- Re-use existing
IndexShardRoutingTable
instances where possible when adjusting the routing table on the elected master - Refine the diffability of the
IndexRoutingTable
so that it avoids sending unchangedIndexShardRoutingTable
instances over the wire.
Relates #77466