[BUG] org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue flaky

[andrross]

See error here: https://build.ci.opensearch.org/job/gradle-check/6130/consoleFull

Not reproducible.

FYI @xuezhou25 This looks like it might be a test you recently introduced?

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue" -Dtests.seed=C7E67525143CA276 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=bg-BG -Dtests.timezone=Pacific/Marquesas -Druntime.java=17

org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT > testIndexDeletionDuringSnapshotCreationInQueue FAILED
    java.lang.IllegalStateException: Cannot cleanup [test-repo] - a snapshot is currently running in [SnapshotsInProgress[test-snap-2]]
        at __randomizedtesting.SeedInfo.seed([C7E67525143CA276:FFCD0E8BC360C7E7]:0)
        at org.opensearch.action.admin.cluster.repositories.cleanup.TransportCleanupRepositoryAction$2.execute(TransportCleanupRepositoryAction.java:254)
        at org.opensearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:65)
        at org.opensearch.cluster.service.MasterService.executeTasks(MasterService.java:843)
        at org.opensearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:414)
        at org.opensearch.cluster.service.MasterService.runTasks(MasterService.java:285)
        at org.opensearch.cluster.service.MasterService$Batcher.run(MasterService.java:196)
        at org.opensearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:176)
        at org.opensearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:214)
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
        at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:282)
        at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:245)
        at java.****/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.****/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.****/java.lang.Thread.run(Thread.java:833)

[dblock]

#5141

[dreamer-89]

Curious on this test failure as it is the second top contributor to gradle check failures in last two months.

[mch2]

#6015