Skip to content

[CI] MultiClusterSearchYamlTestSuiteIT test failures: _shards.total didn't match expected value #58345

Closed
@williamrandolph

Description

@williamrandolph

Build scan: https://gradle-enterprise.elastic.co/s/tak2dqnsyoy3u

Repro line:

REPRODUCE WITH: gradlew ':qa:multi-cluster-search:mixedClusterTestRunner' --tests "org.elasticsearch.search.MultiClusterSearchYamlTestSuiteIT.test {yaml=multi_cluster/*}" \
  -Dtests.seed=E65AA623CD0A6CA2 \
  -Dtests.security.manager=true \
  -Dtests.locale=ms-MY \
  -Dtests.timezone=America/Buenos_Aires \
  -Druntime.java=8 \
  -Dtests.rest.suite=multi_cluster

Reproduces locally?: Yes, but so far only with specific seeds.

Applicable branches: 7.x, master

Failure history:
One release build and three PRs in the last two days. Nothing earlier. See gradle enterprise test history

Failure excerpt:

java.lang.AssertionError: Failure at [multi_cluster/80_index_name_agg:45]: _shards.total didn't match expected value: |  
-- | --
  | _shards.total: expected Integer [2] but was Integer [3] |  
  | at __randomizedtesting.SeedInfo.seed([11FAA49E7D1823A6:99AE9B44D3E44E5E]:0) |  
  | at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:405) |  
  | at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:382) |  
  | at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) |  
  | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) |  
  | at java.lang.reflect.Method.invoke(Method.java:498) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) |  
  | at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) |  
  | at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) |  
  | at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) |  
  | at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) |  
  | at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) |  
  | at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) |  
  | at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) |  
  | at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) |  
  | at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) |  
  | at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) |  
  | at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) |  
  | at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) |  
  | at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) |  
  | at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) |  
  | at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) |  
  | at java.lang.Thread.run(Thread.java:748) |  
  |   |  
  | Caused by: |  
  | java.lang.AssertionError: _shards.total didn't match expected value: |  
  | _shards.total: expected Integer [2] but was Integer [3] |  
  | at org.elasticsearch.test.rest.yaml.section.MatchAssertion.doAssert(MatchAssertion.java:93) |  
  | at org.elasticsearch.test.rest.yaml.section.Assertion.execute(Assertion.java:76) |  
  | at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:398) |  
  | ... 37 more

Timing-wise, this might have to do with #58298, but I can't say for sure. I haven't had much luck digging into this one.

I will mute the failing tests here.

Metadata

Metadata

Assignees

Labels

:Search/SearchSearch-related issues that do not fall into other categories>test-failureTriaged test failures from CITeam:SearchMeta label for search teamv7.9.0v8.0.0-alpha1

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions