Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move JOIN ON predicates push down logic from planner to optimizer #2619

Closed
korowa opened this issue May 25, 2022 · 1 comment · Fixed by #2647
Closed

Move JOIN ON predicates push down logic from planner to optimizer #2619

korowa opened this issue May 25, 2022 · 1 comment · Fixed by #2647
Labels
datafusion Changes in the datafusion crate enhancement New feature or request

Comments

@korowa
Copy link
Contributor

korowa commented May 25, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
At this moment pushdown logic for JOIN ON predicates is provided by logical planner. The idea is to move it to filter_push_down optimizer rule and unify join planning logic.

Describe the solution you'd like
Logical planner should not parse predicates in ON clause (except for join keys) - it should simply put them into Join.filter field. filter_push_down should be able to move JOIN ON predicates (or its parts) as filters over join inputs. After that we could enable ignored tests in filter_push_down.rs related to join filters push down logic.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
PR discussion comment on changing planner logic

PR discussion comment on filter pushdown optimizer

@alamb
Copy link
Contributor

alamb commented May 25, 2022

Another benefit of pushing on predicates in the optimizer is that then the optimization will be done for DataFrame plans as well as SQL

@alamb alamb added the datafusion Changes in the datafusion crate label May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants