Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Concurrent Segment Search][BUG] Incorrect slice collectors to leaves association with profile enabled #10767

Closed
ticheng-aws opened this issue Oct 20, 2023 · 0 comments · Fixed by #11134
Assignees
Labels
bug Something isn't working Search Search query, autocomplete ...etc v2.12.0 Issues and PRs related to version 2.12.0 v3.0.0 Issues and PRs related to version 3.0.0

Comments

@ticheng-aws
Copy link
Contributor

ticheng-aws commented Oct 20, 2023

Describe the bug
In the concurrent search case, we found there is no associateCollectorToLeaves() called for FilteredCollector:getLeafCollector() to build the scorer. This issue caused the issue to create profile result.

ContextIndexSearcher.searchLeaf() collector.getLeafCollector() -> FiliteredCollectorManager.newCollector() -> FilteredCollector.getLeafCollector() -> scorerSupplier(context)

To Reproduce

REPRODUCE WITH: ./gradlew 'null' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testIndexSortingEarlyTermination [p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@487db668]" -Dtests.seed=82745E9780FC847D -Dtests.security.manager=false -Dtests.locale=fr-FR -Dtests.timezone=Kwajalein -Druntime.java=18
java.lang.AssertionError: OpenSearchException[Unexpected size: 6 of leaf breakdown in ConcurrentQueryProfileBreakdown of rewritten query for a leaf.]

	at __randomizedtesting.SeedInfo.seed([82745E9780FC847D:FAAC3E093A92215F]:0)
	at org.opensearch.search.profile.query.ConcurrentQueryProfileBreakdown.toBreakdownMap(ConcurrentQueryProfileBreakdown.java:95)
	at org.opensearch.search.profile.query.ConcurrentQueryProfileTree.createProfileResult(ConcurrentQueryProfileTree.java:44)
	at org.opensearch.search.profile.query.ConcurrentQueryProfileTree.createProfileResult(ConcurrentQueryProfileTree.java:24)
	at org.opensearch.search.profile.AbstractInternalProfileTree.doGetTree(AbstractInternalProfileTree.java:183)
	at org.opensearch.search.profile.AbstractInternalProfileTree.getTree(AbstractInternalProfileTree.java:155)
	at org.opensearch.search.profile.query.ConcurrentQueryProfileTree.getTree(ConcurrentQueryProfileTree.java:77)
	at org.opensearch.search.profile.query.ConcurrentQueryProfiler.getTree(ConcurrentQueryProfiler.java:74)
	at org.opensearch.search.profile.SearchProfileShardResults.buildShardResults(SearchProfileShardResults.java:202)
	at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1494)
	at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1483)
	at org.opensearch.search.query.QueryProfilePhaseTests.testIndexSortingEarlyTermination(QueryProfilePhaseTests.java:857)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: OpenSearchException[Unexpected size: 6 of leaf breakdown in ConcurrentQueryProfileBreakdown of rewritten query for a leaf.]
	... 49 more

Expected behavior
A clear and concise description of what you expected to happen.

Plugins
Standard

Screenshots
N/A

Host/Environment (please complete the following information):

  • CI

Additional context
Add any other context about the problem here.

@ticheng-aws ticheng-aws added bug Something isn't working untriaged labels Oct 20, 2023
@ticheng-aws ticheng-aws changed the title [Concurrent Segment Search][BUG] Incorrect slice collectors to leave association with profile enabled [Concurrent Segment Search][BUG] Incorrect slice collectors to leaves association with profile enabled Oct 31, 2023
@reta reta added v3.0.0 Issues and PRs related to version 3.0.0 Search Search query, autocomplete ...etc v2.12.0 Issues and PRs related to version 2.12.0 and removed untriaged labels Nov 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Search Search query, autocomplete ...etc v2.12.0 Issues and PRs related to version 2.12.0 v3.0.0 Issues and PRs related to version 3.0.0
Projects
Status: Done
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants