Skip to content

Fix a flaky test for issue 19915 #20253

Merged
andrross merged 1 commit intoopensearch-project:mainfrom
liuguoqingfz:flakytese-19915-testUpgradesLegitimateVersions
Dec 17, 2025
Merged

Fix a flaky test for issue 19915 #20253
andrross merged 1 commit intoopensearch-project:mainfrom
liuguoqingfz:flakytese-19915-testUpgradesLegitimateVersions

Conversation

@liuguoqingfz
Copy link
Contributor

@liuguoqingfz liuguoqingfz commented Dec 16, 2025

Description

Fix 2 tests where the failure is coming from NodeMetadataTests.randomVersion() generating an invalid raw version id and then calling Version.fromId(...). Version.fromId(...) now enforces that ids carry the OpenSearch mask, so random integers will intermittently blow up.

Both org.opensearch.env.NodeMetadataTests.testUpgradesLegitimateVersions and

org.opensearch.env.NodeMetadataTests.testEqualsHashcodeSerialization should be fixed by this change

Related Issues

Resolves #19915

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Summary by CodeRabbit

  • Tests
    • Updated version generation validation logic in the test suite to ensure proper version bounds handling.

✏️ Tip: You can customize this high-level summary in your review settings.

…own Version constants which are guaranteed to be valid/masked.

Signed-off-by: Joe Liu <guoqing4@illinois.edu>
@liuguoqingfz liuguoqingfz requested a review from a team as a code owner December 16, 2025 11:22
@github-actions github-actions bot added >test-failure Test failure from CI, local build, etc. autocut flaky-test Random test failure that succeeds on second run labels Dec 16, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 16, 2025

Walkthrough

The randomVersion() method in NodeMetadataTests is updated to generate Version objects using Version.fromId() with an id bounded to the valid masked range (MASK to 234217727) in the rarely() branch. A clarifying comment explains the masking requirement. The other branch retains VersionUtils.randomVersion(). No control flow or error handling changes.

Changes

Cohort / File(s) Summary
Test Flakiness Fix
server/src/test/java/org/opensearch/env/NodeMetadataTests.java
Updated randomVersion() method to generate Version objects within valid masked ranges; added clarifying comment block explaining masking requirement and upper bound rationale

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

  • Verify the MASK and 234217727 bounds align with Version masking requirements
  • Confirm the rarely() branch logic change resolves the reported flaky test failures
  • Ensure the comment accurately reflects the version generation constraints

Suggested labels

bug, flaky-test

Suggested reviewers

  • msfroh
  • cwperks
  • dbwiddis
  • sachinpkale
  • andrross

Poem

🐰 A flaky test once made us quite blue,
With random versions causing trouble through and through,
But now with masks and bounds held tight,
Our versions generate just right—no more fright! ✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the primary change: fixing a flaky test related to issue 19915, with specific focus on the randomVersion() method fix.
Description check ✅ Passed The description explains the root cause of flaky tests, identifies the affected test methods, and references the related issue #19915. It follows the template structure with clear sections.
Linked Issues check ✅ Passed The PR directly addresses the failing tests identified in issue #19915 by fixing the randomVersion() method to generate valid masked version IDs, resolving the flakiness in testUpgradesLegitimateVersions and testEqualsHashcodeSerialization.
Out of Scope Changes check ✅ Passed All changes are focused on fixing the randomVersion() method in NodeMetadataTests.java to resolve the flaky test issue, with no unrelated modifications detected.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
server/src/test/java/org/opensearch/env/NodeMetadataTests.java (1)

54-55: Consider extracting the magic number as a named constant.

The value 234217727 appears both here and in tooNewVersion() (line 121). Extracting it as a named constant (e.g., MAX_TEST_VERSION_ID) would improve maintainability and make the intent clearer.

Example:

+    private static final int MAX_TEST_VERSION_ID = 234217727;
+
     private Version randomVersion() {
         // VersionUtils.randomVersion() only returns known versions, which are necessarily no later than Version.CURRENT; however we want
         // also to consider our behaviour with all versions, so occasionally pick up a truly random version.
         //
         // Version.fromId(...) requires the OpenSearch mask to be present, so we must generate ids in the valid masked range.
-        // The maximum id used elsewhere in this test class is 234217727 (see tooNewVersion()).
-        return rarely() ? Version.fromId(between(MASK, 234217727)) : VersionUtils.randomVersion(random());
+        // The maximum id used elsewhere in this test class is MAX_TEST_VERSION_ID (see tooNewVersion()).
+        return rarely() ? Version.fromId(between(MASK, MAX_TEST_VERSION_ID)) : VersionUtils.randomVersion(random());
     }

Then update line 121 to use the constant as well:

     public static Version tooNewVersion() {
-        return Version.fromId(between(Version.CURRENT.id + 1, 234217727));
+        return Version.fromId(between(Version.CURRENT.id + 1, MAX_TEST_VERSION_ID));
     }
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e798353 and 00e6339.

📒 Files selected for processing (1)
  • server/src/test/java/org/opensearch/env/NodeMetadataTests.java (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
server/src/test/java/org/opensearch/env/NodeMetadataTests.java (1)
test/framework/src/main/java/org/opensearch/test/VersionUtils.java (1)
  • VersionUtils (52-371)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
  • GitHub Check: gradle-check
  • GitHub Check: assemble (25, ubuntu-24.04-arm)
  • GitHub Check: assemble (25, ubuntu-latest)
  • GitHub Check: assemble (25, windows-latest)
  • GitHub Check: assemble (21, ubuntu-latest)
  • GitHub Check: assemble (21, ubuntu-24.04-arm)
  • GitHub Check: assemble (21, windows-latest)
  • GitHub Check: precommit (25, ubuntu-latest)
  • GitHub Check: precommit (25, macos-15-intel)
  • GitHub Check: precommit (25, ubuntu-24.04-arm)
  • GitHub Check: precommit (21, windows-2025, true)
  • GitHub Check: precommit (21, ubuntu-24.04-arm)
  • GitHub Check: precommit (21, windows-latest)
  • GitHub Check: precommit (25, windows-latest)
  • GitHub Check: precommit (21, macos-15-intel)
  • GitHub Check: precommit (21, macos-15)
  • GitHub Check: precommit (25, macos-15)
  • GitHub Check: precommit (21, ubuntu-latest)
  • GitHub Check: detect-breaking-change
  • GitHub Check: Analyze (java)
🔇 Additional comments (1)
server/src/test/java/org/opensearch/env/NodeMetadataTests.java (1)

52-55: LGTM! Fix correctly addresses the flaky test issue.

The change ensures that randomly generated version IDs always include the required OpenSearch mask (MASK) and stay within the valid range, preventing Version.fromId(...) from failing. The upper bound 234217727 is consistent with tooNewVersion() (line 121), and the added comments clearly explain the rationale.

@github-actions
Copy link
Contributor

✅ Gradle check result for 00e6339: SUCCESS

@codecov
Copy link

codecov bot commented Dec 16, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.23%. Comparing base (930ae74) to head (00e6339).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #20253      +/-   ##
============================================
+ Coverage     73.20%   73.23%   +0.02%     
- Complexity    71745    71751       +6     
============================================
  Files          5795     5795              
  Lines        328304   328304              
  Branches      47283    47283              
============================================
+ Hits         240334   240432      +98     
+ Misses        68663    68542     -121     
- Partials      19307    19330      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andrross andrross merged commit f31300f into opensearch-project:main Dec 17, 2025
39 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autocut flaky-test Random test failure that succeeds on second run skip-changelog >test-failure Test failure from CI, local build, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[AUTOCUT] Gradle Check Flaky Test Report for NodeMetadataTests

2 participants