Enhance shrink action in ILM to specify max single primary shard size

[gaobinlong]

Elasticsearch version: master

Plugins installed: []

JVM version (java -version): 14.0.1

OS version (uname -a if on a Unix-like system): Mac OS 10.13.6

Description of the problem including expected versus actual behavior:

Today in the shrink action of ILM, we can only set number_of_shards to a certain value which is a factor of the number of shards in the source index.
However, in our situation, the storage of the source indexs differ from 100GB to 1TB, all of the source indexs contain 60 shards, we want to shrink the soure index according to it's storage, for example, shrink the 100GB index to 2 shards and the 1TB index to 20 shards, ensure that the max primary shard size is 50GB.
So can we add a parameter like max_single_primary_size in the shrink action thus the shard num of the new shrunken index can be calculated by the storage of the source index?

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[tlrx]

We discussed in team and we agreed that the easy path would be to use datastreams with automated max_size rollover using ILM. But we also agreed that it is not always possible to use datastreams and/or timestamp indices, specially if out-of-order documents are indexed or updated, and as such we think that the feature you are proposing could be useful to Elasticsearch users.

The existing resize action which is used for shrinking or splitting indices already uses the index stats before taking an action. For example, before shrinking an index it verifies if the resulting index will exceed the maximum number of documents per shard. I assume that we could use the index stats to compute an appropriate number of shards that would fit under a maximum limit set by a request parameter.

Would you be interested in working on this?

[gaobinlong]

@tlrx, thanks for your reply, I'm very glad to work on this feature. We have to use a python script to shrink index according to it's storage now, so I think the feature is meaningful if we implement it in ILM.

[tlrx]

so I think the feature is meaningful if we implement it in ILM.

I think the feature itself can be implemented in the resize action itself and exposed in ILM.

[jakelandis]

This is somewhat related to #63026 (except that issue is for rollover) as well as #63872.

+1 to general direction.

please cc @elastic/es-core-features for any reviews.