-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce OR Predicate Execution On Star Tree Index #7184
Conversation
This commit allows OR predicates to be executed on Star Tree indices. We only allow a single dimension to be present in the OR predicate in order to be qualified for execution on Star Tree node. This is to ensure that we do not double count pre aggregated documents while traversing the tree for two dimensions. A follow up PR will be done for multi dimensional OR predicates.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filter we want to support here could be nested OR under AND, e.g. a > 10 AND (b < 0 OR b > 10) AND (c < 5 OR c > 20)
. We should not assume the filter only contain one of AND and OR
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
The integration test is failing due to a Kafka issue -- unrelated IMO to this change |
@Jackie-Jiang Updated, please see |
@atris The test failure is in star-tree cluster integration test, which is related to the change in this PR. PTAL |
pinot-core/src/main/java/org/apache/pinot/core/startree/CompositePredicate.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/StarTreeUtils.java
Outdated
Show resolved
Hide resolved
pinot-core/src/main/java/org/apache/pinot/core/startree/operator/StarTreeFilterOperator.java
Outdated
Show resolved
Hide resolved
Thanks, fixed. @Jackie-Jiang Updated the PR, please see and let me know your comments and feedback. |
Codecov Report
@@ Coverage Diff @@
## master #7184 +/- ##
============================================
- Coverage 73.51% 65.27% -8.25%
Complexity 92 92
============================================
Files 1506 1508 +2
Lines 73832 73939 +107
Branches 10655 10682 +27
============================================
- Hits 54281 48262 -6019
- Misses 16011 22260 +6249
+ Partials 3540 3417 -123
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
This commit allows OR predicates to be executed on Star Tree indices. We only allow a single dimension to
be present in the OR predicate in order to be qualified for execution on Star Tree node. This is to ensure
that we do not double count pre aggregated documents while traversing the tree for two dimensions. A follow
up PR will be done for multi dimensional OR predicates.