[CI] ClusterStateWaitThresholdBreachTests testWaitInShrunkShardsAllocatedExceedsThreshold failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #7440 / openjdk21_checkpart2_java-fips-matrix
• elasticsearch-pull-request #66374 / part-2
• elasticsearch-pull-request #66143 / part-2-fips

Reproduction Line:

./gradlew ":x-pack:plugin:ilm:internalClusterTest" --tests "org.elasticsearch.xpack.ilm.ClusterStateWaitThresholdBreachTests.testWaitInShrunkShardsAllocatedExceedsThreshold" -Dtests.seed=2BDAD0719ABA2123 -Dtests.locale=shi-MA -Dtests.timezone=Israel -Druntime.java=21 -Dtests.fips.enabled=true

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

org.elasticsearch.index.IndexNotFoundException: no such index [shrink-cncc-index-rfqmtvnuxfdjt]

Issue Reasons:

• [main] 3 failures in test testWaitInShrunkShardsAllocatedExceedsThreshold (1.0% fail rate in 296 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.7% fail rate in 119 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 3 failures in test testWaitInShrunkShardsAllocatedExceedsThreshold (1.0% fail rate in 296 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.7% fail rate in 119 executions)

Build Scans:

• elasticsearch-periodic #7440 / openjdk21_checkpart2_java-fips-matrix
• elasticsearch-pull-request #66374 / part-2
• elasticsearch-pull-request #66143 / part-2-fips

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[nielsbauman]

This was caused by the index existence check being run on the local node, causing the following update settings API to fail if it the master had not yet finished publishing the new cluster state. Caused by #125652. However, #126501 already fixed this so I'll put up a PR to unmute this test.