Closed
Description
This test has been observed to fail occasionally in the feature-jindex-master branch. I have not yet managed to reproduce locally. I will shortly be muting the test as we need a green build to merge the branch in master. However, I am raising the issue to ensure we get to the bottom of this failure.
Link to failure (one of them): https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request-2/2556/console
Reproduce with:
./gradlew :x-pack:plugin:ml:internalClusterTest -Dtests.seed=D2A618A38265651F -Dtests.class=org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT -Dtests.method="testLoseDedicatedMasterNode" -Dtests.security.manager=true -Dtests.locale=be-BY -Dtests.timezone=Asia/Katmandu -Dcompiler.java=11 -Druntime.java=8
Failure:
> Throwable #1: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:847)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:821)
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.run(MlDistributedFailureIT.java:292)
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.testLoseDedicatedMasterNode(MlDistributedFailureIT.java:88)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 more
> Suppressed: java.lang.AssertionError
> at org.elasticsearch.xpack.ml.integration.MlDistributedFailureIT.lambda$run$15(MlDistributedFailureIT.java:298)
> at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
> ... 40 moreThrowable #2: java.lang.RuntimeException: Had to resort to force-stopping datafeed, something went wrong?
> at org.elasticsearch.xpack.ml.support.BaseMlIntegTestCase.deleteAllDatafeeds(BaseMlIntegTestCase.java:296)
> at org.elasticsearch.xpack.ml.support.BaseMlIntegTestCase.cleanupWorkaround(BaseMlIntegTestCase.java:209)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.ExecutionException: ElasticsearchStatusException[Cannot stop datafeed [data_feed_id] because the datafeed does not have an assigned node. Use force stop to stop the datafeed]
> at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.getValue(BaseFuture.java:266)
> at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:253)
> at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:87)
> at org.elasticsearch.xpack.ml.support.BaseMlIntegTestCase.deleteAllDatafeeds(BaseMlIntegTestCase.java:284)
> ... 36 more
> Caused by: ElasticsearchStatusException[Cannot stop datafeed [data_feed_id] because the datafeed does not have an assigned node. Use force stop to stop the datafeed]
> at org.elasticsearch.xpack.core.ml.utils.ExceptionsHelper.conflictStatusException(ExceptionsHelper.java:50)
> at org.elasticsearch.xpack.ml.action.TransportStopDatafeedAction.normalStopDatafeed(TransportStopDatafeedAction.java:147)
> at org.elasticsearch.xpack.ml.action.TransportStopDatafeedAction.lambda$doExecute$0(TransportStopDatafeedAction.java:130)
> at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60)
> at org.elasticsearch.xpack.ml.datafeed.persistence.DatafeedConfigProvider.lambd