Skip to content

[ILM] Avoid race condition between shrinking and ILM itself #109206

Open

Description

In the ILM Shrink action, ILM performs (among others) the following steps:

StepKey shrinkKey = new StepKey(phase, NAME, ShrinkStep.NAME);
StepKey enoughShardsKey = new StepKey(phase, NAME, ShrunkShardsAllocatedStep.NAME);
StepKey copyMetadataKey = new StepKey(phase, NAME, CopyExecutionStateStep.NAME);

The ShrinkStep copies the ILM policy name in the shrink request:

However, when the allocation of the shards of the shrunken index takes "some" time (which is not unexpected for larger shards), which is what the second step waits for, ILM starts executing the policy on the shrunken index before the ILM execution state has been copied to that index (step three). This means ILM will start executing the policy from the beginning for the shrunken index, causing the WaitForRolloverReadyStep to fail here, instead of proceeding in the shrink action.

This is not necessarily harmful, it's just "noisy" - there is no data loss, it only produces some error logs and possibly a yellow health API status.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions