Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce OR Predicate Execution On Star Tree Index #7184

Merged
merged 8 commits into from
Jul 27, 2021

Conversation

atris
Copy link
Contributor

@atris atris commented Jul 21, 2021

This commit allows OR predicates to be executed on Star Tree indices. We only allow a single dimension to
be present in the OR predicate in order to be qualified for execution on Star Tree node. This is to ensure
that we do not double count pre aggregated documents while traversing the tree for two dimensions. A follow
up PR will be done for multi dimensional OR predicates.

This commit allows OR predicates to be executed on Star Tree indices. We only allow a single dimension to
be present in the OR predicate in order to be qualified for execution on Star Tree node. This is to ensure
that we do not double count pre aggregated documents while traversing the tree for two dimensions. A follow
up PR will be done for multi dimensional OR predicates.
Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter we want to support here could be nested OR under AND, e.g. a > 10 AND (b < 0 OR b > 10) AND (c < 5 OR c > 20). We should not assume the filter only contain one of AND and OR

@atris
Copy link
Contributor Author

atris commented Jul 23, 2021

The integration test is failing due to a Kafka issue -- unrelated IMO to this change

@atris
Copy link
Contributor Author

atris commented Jul 23, 2021

@Jackie-Jiang Updated, please see

@atris atris requested a review from Jackie-Jiang July 23, 2021 18:11
@Jackie-Jiang
Copy link
Contributor

@atris The test failure is in star-tree cluster integration test, which is related to the change in this PR. PTAL

@atris
Copy link
Contributor Author

atris commented Jul 24, 2021

@atris The test failure is in star-tree cluster integration test, which is related to the change in this PR. PTAL

Thanks, fixed.

@Jackie-Jiang Updated the PR, please see and let me know your comments and feedback.

@codecov-commenter
Copy link

Codecov Report

Merging #7184 (4a51773) into master (fe83e95) will decrease coverage by 8.24%.
The diff coverage is 68.45%.

❗ Current head 4a51773 differs from pull request most recent head f7d6bf4. Consider uploading reports for the commit f7d6bf4 to get more accurate results
Impacted file tree graph

@@             Coverage Diff              @@
##             master    #7184      +/-   ##
============================================
- Coverage     73.51%   65.27%   -8.25%     
  Complexity       92       92              
============================================
  Files          1506     1508       +2     
  Lines         73832    73939     +107     
  Branches      10655    10682      +27     
============================================
- Hits          54281    48262    -6019     
- Misses        16011    22260    +6249     
+ Partials       3540     3417     -123     
Flag Coverage Δ
integration ?
unittests 65.27% <68.45%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../main/java/org/apache/pinot/client/Connection.java 34.04% <0.00%> (-9.44%) ⬇️
...n/java/org/apache/pinot/client/ExecutionStats.java 68.88% <ø> (ø)
.../org/apache/pinot/client/ResultTableResultSet.java 88.00% <ø> (ø)
.../org/apache/pinot/client/SimpleBrokerSelector.java 83.33% <0.00%> (-16.67%) ⬇️
.../org/apache/pinot/core/common/MinionConstants.java 0.00% <0.00%> (ø)
...t/core/plan/AggregationGroupByOrderByPlanNode.java 45.00% <0.00%> (ø)
...he/pinot/core/plan/AggregationGroupByPlanNode.java 42.10% <0.00%> (-42.11%) ⬇️
...rg/apache/pinot/core/plan/AggregationPlanNode.java 36.66% <0.00%> (-50.00%) ⬇️
...re/segment/processing/framework/SegmentConfig.java 87.50% <ø> (ø)
...t/core/startree/plan/StarTreeDocIdSetPlanNode.java 0.00% <0.00%> (-100.00%) ⬇️
... and 393 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe83e95...f7d6bf4. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants