-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] flaky test index/80_geo_point/Single point test
#4852
Comments
Another PR impacted by this - #4805 |
Test consistently failing for #4870 |
Fixed by #4860 |
It looks like the test has been disabled - yay for not failing gradle checks! - but it has not been fixed. I think this issue should stay open until the test is fixed or deleted. |
The test is not disabled. The confusion is coming from the naming of the test. The test is newly added with new feature of geojson format support for point type in version 2.4. Therefore, it fails with previous version which does not support the geojson format for point type. Let me separate the two cases out instead of combining it... |
@heemin32 Good detail! I am curious if the test was running against versions of OpenSearch that did not support the feature, then I would expect gradle check to be blocked since the test was merged - do you know how the test presented itself as intermittently failing? |
@peternied My understanding is that in the mixed cluster test case the requests may go to any node. If the requests requiring the new feature happen to hit only the nodes with the newer version of OpenSearch, then the tests will pass. Hence in the case of a compatibility issue you might see intermittent successes and failures.
@heemin32 What action are you taking here? |
@andrross Creating test case for testing all previous formats and creating another test case testing geojson format for only version 2.4 and above. Do you think we need that? |
I am going to close out this issue since its tracking the test failure itself, up to @heemin32 if you'd like to re-work the test case to be more obvious about its dependencies.
This sounds like tests are inherently non-deterministic - seems like something we should try to fix as it might be the root cause of many other intermittent failures. @andrross Could you open an issue to track separately? |
@peternied Opened an issue here #5257 |
Describe the bug
Looks like there is a flaky test case
index/80_geo_point/Single point test
inside of org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.testTo Reproduce
Steps to reproduce the behavior:
REPRODUCE WITH: ./gradlew ':qa:mixed-cluster:v2.4.0#mixedClusterTest' --tests "org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=index/80_geo_point/Single point test}" -Dtests.seed=9E49FBE4E90B87AB -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-AR -Dtests.timezone=Asia/Tehran -Druntime.java=17
Expected behavior
There should be no flaky tests in the gradle check
Host/Environment (please complete the following information):
Additional context
Please react with 🤔 if you see further instances and paste a link to the impacted pull request in this issue.
The text was updated successfully, but these errors were encountered: