[CI] org.elasticsearch.percolator.CandidateQueryTests.testDuel2 failure

[iverase]

This test seems to have been failing recently:

master: https://gradle-enterprise.elastic.co/s/eel3j36akwpam
7.x: https://gradle-enterprise.elastic.co/s/xjdre63zy2hmk

It does not reproduce locally. Failure:

org.elasticsearch.percolator.CandidateQueryTests > testDuel2 FAILED
--
java.lang.AssertionError:
Expected: <6L>
but: was <5L>
at __randomizedtesting.SeedInfo.seed([D22432087DDAD7A4:E884A4C0AB3467C]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at org.elasticsearch.percolator.CandidateQueryTests.duelRun(CandidateQueryTests.java:1062)
at org.elasticsearch.percolator.CandidateQueryTests.testDuel2(CandidateQueryTests.java:376

[elasticmachine]

Pinging @elastic/es-search (:Search/Percolator)

[cbuescher]

Wasn't able to reproduce locally either, but for anybody else giving it a try, here's the REPRODUCE line:

REPRODUCE WITH: ./gradlew ':modules:percolator:test' --tests "org.elasticsearch.percolator.CandidateQueryTests.testDuel2" -Dtests.seed=957B0F4D9C5DCC90 -Dtests.security.manager=true -Dtests.locale=fi-FI -Dtests.timezone=Africa/Mbabane -Dcompiler.java=12

Some details from the log that show the actual and the control documents:

1> [2019-11-28T06:13:03,870][INFO ][o.e.c.m.MetaDataCreateIndexService] [node_s_0] [test] creating index, cause [api], templates [one_shard_index_template, random-soft-deletes-template], shards [1]/[0], mappings []
--
1> [2019-11-28T06:13:03,989][INFO ][o.e.c.r.a.AllocationService] [node_s_0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[test][0]]]).
1> [2019-11-28T06:13:04,100][INFO ][o.e.p.CandidateQueryTests] [testDuel2] Test with document: {}Document<indexed,tokenized<string_field:value1>>
1> [2019-11-28T06:13:04,114][INFO ][o.e.p.CandidateQueryTests] [testDuel2] Test with document: {}Document<indexed,tokenized<string_field:value1> indexed,tokenized<string_field:value2>>
1> [2019-11-28T06:13:04,131][INFO ][o.e.p.CandidateQueryTests] [testDuel2] Test with document: {}Document<indexed,tokenized<string_field:value1> indexed,tokenized<string_field:value2> indexed,tokenized<string_field:value3>>
1> [2019-11-28T06:13:04,159][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.totalHits=6 hits
1> [2019-11-28T06:13:04,159][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.totalHits=7 hits
1> [2019-11-28T06:13:04,160][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs.length=6
1> [2019-11-28T06:13:04,160][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs.length=7
1> [2019-11-28T06:13:04,160][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[0].doc=9
1> [2019-11-28T06:13:04,161][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[0].score=1.0
1> [2019-11-28T06:13:04,161][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[1].doc=1
1> [2019-11-28T06:13:04,161][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[1].score=0.39229375
1> [2019-11-28T06:13:04,161][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[2].doc=5
1> [2019-11-28T06:13:04,161][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[2].score=0.39229372
1> [2019-11-28T06:13:04,162][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[3].doc=7
1> [2019-11-28T06:13:04,162][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[3].score=0.26152915
1> [2019-11-28T06:13:04,162][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[4].doc=0
1> [2019-11-28T06:13:04,162][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[4].score=0.13076457
1> [2019-11-28T06:13:04,162][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[5].doc=6
1> [2019-11-28T06:13:04,163][ERROR][o.e.p.CandidateQueryTests] [testDuel2] topDocs.scoreDocs[5].score=0.13076457
1> [2019-11-28T06:13:04,163][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[0].doc=9
1> [2019-11-28T06:13:04,165][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[0].score=1.0
1> [2019-11-28T06:13:04,167][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[0].query_to_string=*:*
1> [2019-11-28T06:13:04,170][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[0].query_terms_field=
1> [2019-11-28T06:13:04,174][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[0].minimum_should_match_field=0
1> [2019-11-28T06:13:04,174][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[1].doc=1
1> [2019-11-28T06:13:04,175][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[1].score=0.39229375
1> [2019-11-28T06:13:04,175][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[1].query_to_string=string_field:"value1 value2 value3"
1> [2019-11-28T06:13:04,176][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[1].query_terms_field=string_fieldvalue1,string_fieldvalue2,string_fieldvalue3,
1> [2019-11-28T06:13:04,176][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[1].minimum_should_match_field=3
1> [2019-11-28T06:13:04,176][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[2].doc=5
1> [2019-11-28T06:13:04,176][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[2].score=0.39229372
1> [2019-11-28T06:13:04,177][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[2].query_to_string=(((((string_field:value2 int_field:[15 TO 50] ((string_field:value3)~1) (string_field:value2))~2))~1) (((((#string_field:value2) ((int_field:[15 TO 50] string_field:value1 string_field:value3 string_field:value1)~4) +int_field:[0 TO 10])~2))~1))~1
1> [2019-11-28T06:13:04,177][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[2].query_terms_field=string_fieldvalue1,string_fieldvalue2,string_fieldvalue3,
1> [2019-11-28T06:13:04,177][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[2].minimum_should_match_field=1
1> [2019-11-28T06:13:04,178][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[3].doc=7
1> [2019-11-28T06:13:04,178][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[3].score=0.26152915
1> [2019-11-28T06:13:04,178][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[3].query_to_string=(string_field:value2 (#((int_field:[15 TO 50]) ((string_field:value1 int_field:[15 TO 50] string_field:value1 string_field:value2)~4)) +((int_field:[0 TO 10])~1)) (+(+(string_field:value2))))~1
1> [2019-11-28T06:13:04,178][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[3].query_terms_field=string_fieldvalue1,string_fieldvalue2,
1> [2019-11-28T06:13:04,178][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[3].minimum_should_match_field=1
1> [2019-11-28T06:13:04,179][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[4].doc=0
1> [2019-11-28T06:13:04,179][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[4].score=0.13076457
1> [2019-11-28T06:13:04,179][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[4].query_to_string=string_field:value1
1> [2019-11-28T06:13:04,179][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[4].query_terms_field=string_fieldvalue1,
1> [2019-11-28T06:13:04,179][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[4].minimum_should_match_field=1
1> [2019-11-28T06:13:04,180][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[5].doc=4
1> [2019-11-28T06:13:04,180][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[5].score=0.13076457
1> [2019-11-28T06:13:04,180][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[5].query_to_string=(string_field:value3 int_field:[0 TO 10] #string_field:value3)~1
1> [2019-11-28T06:13:04,180][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[5].query_terms_field=string_fieldvalue3,
1> [2019-11-28T06:13:04,180][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[5].minimum_should_match_field=2
1> [2019-11-28T06:13:04,181][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[6].doc=6
1> [2019-11-28T06:13:04,181][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[6].score=0.13076457
1> [2019-11-28T06:13:04,181][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[6].query_to_string=(string_field:value2)~1
1> [2019-11-28T06:13:04,181][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[6].query_terms_field=string_fieldvalue2,
1> [2019-11-28T06:13:04,181][ERROR][o.e.p.CandidateQueryTests] [testDuel2] controlTopDocs.scoreDocs[6].minimum_should_match_field=1
1> [2019-11-28T06:13:04,212][INFO ][o.e.p.CandidateQueryTests] [testDuel2] after test

It appears in this case controlTopDocs.scoreDocs[5].doc=4 is the culprit that's either missing or too much when comparing the hit count.

[romseygeek]

The query #((((string_field:value1 int_field:[15 TO 50] string_field:value1 string_field:value2)~2))~1) #string_field:value2 +((+((int_field:[-5 TO 5])~1)) #string_field:value2 +string_field:value1) is being analyzed to require a document to contain string_field:value1 and string_field:value with a minimum should match of 3, which is obviously not possible. Working on a fix now.