-
Couldn't load subscription status.
- Fork 130
Description
Is your feature request related to a problem?
Yes. OpenSearch ISM currently lacks conditions to:
- Determine if an index is still pointed by an alias.
- Control transitions based on the amount of time an index has spent in a particular ISM state.
This limits the ability to express real-world policies like:
- Transitioning to an archive state only after the index is unaliased. This is especially helpful when users query indices via aliases only.
- Delaying deletion for a fixed retention window (e.g., 7 days in
archivebefore transitioning todelete).
Why min_state_age is required even though min_index_age is supported
min_index_age in ISM measures time since the index was created, which can lead to premature transitions if the index enters a new state much later in its lifecycle.
For example, if an index is created and stays in the hot state for some time — say 5 days — before transitioning to archive due to a dynamic condition like no_alias: true or min_doc_count, a transition based on min_index_age: 7d would cause the index to move to delete just 2 days after entering archive. This breaks the expectation that the index should remain in the archive state for a full 7 days. Since min_index_age measures time since index creation and is not tied to how long the index has been in its current state, it cannot enforce state-specific retention periods. In contrast, min_state_age starts counting from the moment the index enters a given state, making it possible to implement accurate, state-specific delays — like “stay in archive for 7 days before deletion” — regardless of when or why the transition to archive happened.
By contrast, min_state_age starts counting from when the index enters the current state, allowing policies to enforce precise retention windows or delay transitions. This enables reliable patterns like “stay in archive for 7 days” or “delay deletion for 24 hours after entering a cold state.”
What solution would you like?
Introduce two new transition conditions in ISM:
- no_alias (boolean)
Enables ISM to transition based on whether an index has any alias.
Example use cases:
no_alias: true: transition only after index is removed from alias (e.g., post-rollover).no_alias: false: transition while alias is still present (e.g., ingest-to-archive pipelines).
- min_state_age (string duration, e.g.,
7d,5m)
- Enables transitions only after an index has spent a minimum time in its current ISM state.
- Useful for policies requiring retention or cooling periods.
These conditions should be:
- Supported in ISM policies as standard transition conditions.
- Visible in ISM explain output.
What alternatives have you considered?
- Manually removing aliases before transitions — error-prone and hard to automate.
- Estimating retention using
min_index_age, which lacks per-state accuracy. - Using external scripts to orchestrate retention windows.
These alternatives are more fragile and require custom operational overhead.
Do you have any additional context?
These features were developed and tested locally. They are intended to make ISM more declarative and better suited to real-world search index lifecycle needs (e.g. retention cleanup, time-delayed archiving).
Here’s an example policy this would support:
{
"policy": {
"default_state": "hot",
"states": [
{
"name": "hot",
"actions": [],
"transitions": [
{
"state_name": "archive",
"conditions": {
"no_alias": true
}
}
]
},
{
"name": "archive",
"actions": [],
"transitions": [
{
"state_name": "delete",
"conditions": {
"min_state_age": "7d"
}
}
]
},
{
"name": "delete",
"actions": [{ "delete": {} }],
"transitions": []
}
]
}
}