[AUTOCUT] Gradle Check Flaky Test Report for WarmIndexSegmentReplicationIT

[opensearch-ci-bot]

Flaky Test Report for WarmIndexSegmentReplicationIT

Noticed the WarmIndexSegmentReplicationIT has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
main	null	57203 60336 60346 60883 61065 61090 62120 62599 63244 63374 63786 64498 65023 66136	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testDropPrimaryDuringReplication org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testIndexReopenClose org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testIndexWhileRecoveringReplica org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testReplicationAfterForceMergeOnPrimaryShardsOnly org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testCancellation org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testPrimaryStopped_ReplicaPromoted org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testRealtimeTermVectorRequestsSuccessful org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testReplicaHasDiffFilesThanPrimary org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testScrollWithConcurrentIndexAndSearch org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testShardPathDeletionWhenWarmIndexRelocate
99f3189	19495	64883	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testRestartPrimary
4b9aa27	18550	63886	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testReplicationAfterPrimaryRefreshAndFlush
c56da68	19296	63885	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testShardPathDeletionWhenWarmIndexRelocate
2cec4b5	18646	59878	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testScrollWithConcurrentIndexAndSearch
1adc8cd	18585	59573	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testShardPathDeletionWhenWarmIndexRelocate
cb30055	18060	57402	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testPrimaryStopped_ReplicaPromoted
0795bb2	18099	57396	org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testPrimaryStopped_ReplicaPromoted

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the WarmIndexSegmentReplicationIT class are:

• 17617
• 18147
• 18333
• 17507
• 17902
• 19003
• 18467
• 18565
• 18599
• 19703
• 17556
• 17580
• 17598
• 17606
• 17792
• 17890
• 18069
• 18070
• 18116
• 18166
• 18195
• 18424
• 18573
• 18593
• 18610
• 18621
• 18631
• 18656
• 18664
• 18702
• 18720
• 18749
• 18800
• 18824
• 18826
• 18842
• 18887
• 18890
• 18929
• 18944
• 18951
• 18973
• 19001
• 19007
• 19054
• 19132
• 19152
• 19165
• 19226
• 19278
• 19294
• 19307
• 19387
• 19508
• 19555
• 19576
• 19619
• 19648

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[andrross]

FYI @skumawat2025

[andrross]

https://build.ci.opensearch.org/job/gradle-check/57471/consoleText:

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testPrimaryReceivesDocsDuringReplicaRecovery" -Dtests.seed=C5CF2E73AD7D00AF -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en-PR -Dtests.timezone=Europe/Busingen -Druntime.java=21

WarmIndexSegmentReplicationIT > testPrimaryReceivesDocsDuringReplicaRecovery FAILED
    java.lang.AssertionError: Expected search hits on node: node_t1 to be at least 2 but was: 1
        at __randomizedtesting.SeedInfo.seed([C5CF2E73AD7D00AF:68976B8B8CAA7ED1]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.indices.replication.SegmentReplicationBaseIT.lambda$waitForSearchableDocs$0(SegmentReplicationBaseIT.java:122)
        at org.opensearch.test.OpenSearchTestCase.assertBusyWithFixedSleepTime(OpenSearchTestCase.java:1214)
        at org.opensearch.indices.replication.SegmentReplicationBaseIT.waitForSearchableDocs(SegmentReplicationBaseIT.java:117)
        at org.opensearch.indices.replication.SegmentReplicationBaseIT.waitForSearchableDocs(SegmentReplicationBaseIT.java:112)
        at org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testPrimaryReceivesDocsDuringReplicaRecovery(WarmIndexSegmentReplicationIT.java:1211)

https://build.ci.opensearch.org/job/gradle-check/57472/consoleText:

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testDeleteOperations" -Dtests.seed=3D049AF853D4BA75 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=el-CY -Dtests.timezone=Antarctica/Troll -Druntime.java=21

WarmIndexSegmentReplicationIT > testDeleteOperations FAILED
    java.lang.AssertionError: 
    Expected: a value equal to or greater than <2L>
         but: <0L> was less than <2L>
        at __randomizedtesting.SeedInfo.seed([3D049AF853D4BA75:8E50F0725EFA66E3]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:964)
        at org.junit.Assert.assertThat(Assert.java:930)
        at org.opensearch.test.OpenSearchIntegTestCase.lambda$waitForDocs$3(OpenSearchIntegTestCase.java:1078)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1182)
        at org.opensearch.test.OpenSearchIntegTestCase.waitForDocs(OpenSearchIntegTestCase.java:1045)
        at org.opensearch.indices.replication.WarmIndexSegmentReplicationIT.testDeleteOperations(WarmIndexSegmentReplicationIT.java:794)

[skumawat2025]

Looks like some of the tests are still being flaky - Will take a look and get these fixed.