Skip to content

Conversation

@rgsriram
Copy link
Contributor

Description

The test LeafSorterOptimizationTests.testReadOnlyEngineUsesLeafSorter was flaky in CI due to inconsistent segment creation. We fixed it by indexing more documents, disabling merges, adjusting flush frequency, and making assertions handle both single- and multi-segment cases.

Related Issues

Resolves #18898

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

❌ Gradle check result for 635fd3f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 676e265: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for a4a15c2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 728f806: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 4841617: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

❌ Gradle check result for bf7bd77: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@rgsriram
Copy link
Contributor Author

rgsriram commented Sep 1, 2025

@andrross - Could you please help me here?. I am not getting a detection of breaking changes in the local environment. I would like to know what I am missing here.

./gradlew japicmp                                                                                                                                                                                               
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14.3
  OS Info               : Mac OS X 15.5 (aarch64)
  JDK Version           : 24 (Homebrew JDK)
  JAVA_HOME             : /opt/homebrew/Cellar/openjdk/24.0.2/libexec/openjdk.jdk/Contents/Home
  Random Testing Seed   : E60E942F5983423D
  Crypto Standard       : any-supported
=======================================

BUILD SUCCESSFUL in 6s
35 actionable tasks: 35 up-to-date

Copy link
Member

@andrross andrross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rgsriram I believe the failures here are intermittent failures related to a spotless check:

A problem occurred configuring project ':build-tools'.
> Could not create task ':build-tools:spotlessJava'.
   > java.io.IOException: Failed to load eclipse jdt formatter: java.lang.RuntimeException: java.net.SocketTimeoutException: Connect timed out

Go ahead and push a commit to remove the changelog entry to retry the workflows.

@prudhvigodithi
Copy link
Member

@rgsriram I believe the failures here are intermittent failures related to a spotless check:

A problem occurred configuring project ':build-tools'.
> Could not create task ':build-tools:spotlessJava'.
   > java.io.IOException: Failed to load eclipse jdt formatter: java.lang.RuntimeException: java.net.SocketTimeoutException: Connect timed out

Go ahead and push a commit to remove the changelog entry to retry the workflows.

The fix has been merged #19199, @rgsriram re-base should fix this error. Thanks

…eliability

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2025

❌ Gradle check result for bbb48d4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2025

✅ Gradle check result for f5fa030: SUCCESS

@codecov
Copy link

codecov bot commented Sep 7, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.73%. Comparing base (4aff917) to head (f5fa030).
⚠️ Report is 36 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #19191      +/-   ##
============================================
- Coverage     72.88%   72.73%   -0.15%     
+ Complexity    69650    69620      -30     
============================================
  Files          5658     5658              
  Lines        320085   320085              
  Branches      46345    46345              
============================================
- Hits         233296   232829     -467     
- Misses        67886    68408     +522     
+ Partials      18903    18848      -55     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rgsriram
Copy link
Contributor Author

rgsriram commented Sep 7, 2025

@andrross - Please kindly check.

@msfroh msfroh merged commit 28597dc into opensearch-project:main Sep 18, 2025
31 checks passed
jainankitk pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 22, 2025
Fix flaky LeafSorterOptimizationTests by improving segment creation reliability

---------

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
jainankitk pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 22, 2025
Fix flaky LeafSorterOptimizationTests by improving segment creation reliability

---------

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
Signed-off-by: Ankit Jain <jainankitk@apache.org>
jainankitk pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 22, 2025
Fix flaky LeafSorterOptimizationTests by improving segment creation reliability

---------

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
Signed-off-by: Ankit Jain <jainankitk@apache.org>
asimmahmood1 pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 23, 2025
Fix flaky LeafSorterOptimizationTests by improving segment creation reliability

---------

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
pranikum pushed a commit to pranikum/OpenSearch that referenced this pull request Sep 23, 2025
Fix flaky LeafSorterOptimizationTests by improving segment creation reliability

---------

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
vinaykpud pushed a commit to vinaykpud/OpenSearch that referenced this pull request Sep 26, 2025
Fix flaky LeafSorterOptimizationTests by improving segment creation reliability

---------

Signed-off-by: Sriram Ganesh <srignsh22@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autocut flaky-test Random test failure that succeeds on second run >test-failure Test failure from CI, local build, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[AUTOCUT] Gradle Check Flaky Test Report for LeafSorterOptimizationTests

4 participants