Making shard allocation aware-routing still available in the next major version

[nickcanz]

Describe the feature:

👋 We currently use shard allocation aware routing so that requests can be serviced completely within an "availability-zone". As discussed in #43453 and implemented in #48351, this feature is deprecated now and set to be removed entirely in the next major version of Elasticsearch.

I understand that adaptive replica selection should mostly replace this behavior, but due to cross-zone traffic costs and consistency, I would still strongly prefer a shard allocation aware routing if available.

If possible, I'd be happy with an "opt-in" option that still be available in the next major version of Elasticsearch for searches to be routed based on shard allocation awareness. Another idea would to have more influence on the adaptive replica selection algorithm, to be able to "weight" shard allocation awareness within that logic.

Thanks!

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Distributed)

[elasticmachine]

Pinging @elastic/es-search (:Search/Search)

[jimczi]

Sorry for the late reply @nickcanz. We've discussed this issue internally and have decided to open a new feature request to add a way to prefer nodes within the same availability zone. The feature in 7.x is based on the tags that are set in allocation awareness attributes and search requests only check for at least one attribute in common when performing the routing so it's easy to misconfigure it.
So instead of restoring the old functionality we'd like to discuss better ways to achieve the same without confusing users.
I am closing this issue in favor of #60236 where we'll discuss actual implementations and ideas around this feature.