Add max_primary_shard_size as a rollover action in State Management Policy

[Kevc1]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
We need a way to rollover our indices if the shard size is >50gb as recommended by AWS. This option exists in Elasticsearch and would be very beneficial to have available in Opensearch.

Describe the solution you'd like
A clear and concise description of what you want to happen.
Would like to have this option available in an ISM Policy, such as belwo:
"name": "hot",
"actions": [
{
"rollover": {
"min_size": "90gb",
"min_index_age": "7d",
"max_primary_shard_size" : "50gb"

                    }
                 },
              ]

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[CEHENKLE]

Oops -- Moved wrong issue :/

[elfisher]

@CEHENKLE could you help move this issue to the Index Management repo?

[dbbaughe]

@Kevc1

What is the expectation in terms of evaluating the condition to true:

• When at least one of the primary shards is > 50GB
• When the majority of primary shards are > 50GB
• When all of the primary shards are > 50GB

I'm guessing it's the first one.. but do you see value in the others at all?

[Kevc1]

I was initially thinking the second one but if any shard is going above the limit we probably want to rollover, so #1 would probably be the safest.

[dbbaughe]

@Kevc1, Yes, the second and third one could end up with a hot shards issue where a lot of data is written to specific shards if using something like custom routing, perhaps before the rollover condition is met you have some shards that are already hundreds of GB or more.

[dbbaughe]

Merged into main, will go out in the next release (1.3).