Description
Description
Currently ILM rollover policies allow setting maximum values on age and size of indices/shards that trigger rollovers.
This works well as far as managing the lifecycle of indices that experience non-trivial write load goes.
When managing a large number of indices of varying write-load though, setting maximums only can lead to a lot of rolling over of trivially sized indices that contributes little to moving data volume to colder tiers but costs a lot in terms of the overhead of managing more indices.
Could we add a minimum size requirement for rollovers to the rollover policy as well to add a threshold below which we do not roll over? This way we could run the same policy for data streams that experience high and low write load (e.g. logging data streams for heterogeneous producers) in a way that allows for daily rollover for high volume producers that produce tens of GB a day and avoid the needless overhead of rolling over indices that only see an influx of tens of MB daily.
cc @jpountz