Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor query rewriter to interfaces and implementations #7576

Merged
merged 1 commit into from
Oct 19, 2021

Conversation

xiangfu0
Copy link
Contributor

@xiangfu0 xiangfu0 commented Oct 14, 2021

Description

  • Refactor query rewriter to interfaces
  • Move existing query rewriters to implementations
    • AliasApplier
    • CompileTimeFunctionsInvoker
    • NonAggregationGroupByToDistinctQueryRewriter
    • OrdinalsUpdater
    • PredicateComparisonRewriter
    • SelectionsRewriter
  • QueryVisistors are constructed from a list of class names.
  • Add an environment variable queryVisitorsClassNames to allow override the default query rewriter class names.

Upgrade Notes

Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

  • Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

  • Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

  • New configuration options
  • Deprecation of configurations
  • Signature changes to public methods/interfaces
  • New plugins added or old plugins removed
  • Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes

Documentation

@xiangfu0 xiangfu0 force-pushed the calcite-parser-optimizer branch 2 times, most recently from 6b31182 to 8c91a2a Compare October 14, 2021 07:17
@codecov-commenter
Copy link

codecov-commenter commented Oct 14, 2021

Codecov Report

Merging #7576 (a008154) into master (aed6ada) will decrease coverage by 7.37%.
The diff coverage is 92.73%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #7576      +/-   ##
============================================
- Coverage     71.61%   64.23%   -7.38%     
+ Complexity     3884     3804      -80     
============================================
  Files          1552     1550       -2     
  Lines         79012    78705     -307     
  Branches      11705    11668      -37     
============================================
- Hits          56585    50559    -6026     
- Misses        18618    24549    +5931     
+ Partials       3809     3597     -212     
Flag Coverage Δ
integration1 29.42% <73.35%> (+<0.01%) ⬆️
integration2 27.92% <73.35%> (+0.05%) ⬆️
unittests1 68.58% <92.50%> (-0.01%) ⬇️
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...va/org/apache/pinot/spi/utils/CommonConstants.java 26.31% <ø> (ø)
.../parsers/rewriter/PredicateComparisonRewriter.java 84.09% <84.09%> (ø)
...he/pinot/sql/parsers/rewriter/OrdinalsUpdater.java 90.00% <90.00%> (ø)
...pinot/sql/parsers/rewriter/SelectionsRewriter.java 90.00% <90.00%> (ø)
.../parsers/rewriter/CompileTimeFunctionsInvoker.java 90.47% <90.47%> (ø)
...not/sql/parsers/rewriter/QueryRewriterFactory.java 91.66% <91.66%> (ø)
...pache/pinot/sql/parsers/rewriter/AliasApplier.java 97.14% <97.14%> (ø)
...e/pinot/broker/broker/helix/BaseBrokerStarter.java 71.66% <100.00%> (-1.93%) ⬇️
...org/apache/pinot/sql/parsers/CalciteSqlParser.java 91.39% <100.00%> (-0.26%) ⬇️
.../NonAggregationGroupByToDistinctQueryRewriter.java 100.00% <100.00%> (ø)
... and 218 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aed6ada...a008154. Read the comment docs.

Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! It makes the parser much cleaner



public interface QueryVisitor {
void visit(PinotQuery pinotQuery);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add some javadoc for the class and method?
For the rewrite, should we return a PinotQuery or always rewrite in-place? For now in-place is enough, not sure about the future rewriters

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, since we are not following visitor model, then I can change the return type to PinotQuery, it's up to each implementation to decide if it's in-place or create new.

@xiangfu0 xiangfu0 force-pushed the calcite-parser-optimizer branch 7 times, most recently from 55df822 to f8ce84c Compare October 15, 2021 17:14
@xiangfu0 xiangfu0 requested a review from Jackie-Jiang October 15, 2021 21:14
Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM otherwise

@xiangfu0 xiangfu0 force-pushed the calcite-parser-optimizer branch 7 times, most recently from b6bc8d5 to 6368eec Compare October 19, 2021 20:30
kriti-sc pushed a commit to kriti-sc/incubator-pinot that referenced this pull request Dec 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants