Skip to content

Conversation

@peter-toth
Copy link
Contributor

Which issue does this PR close?

Part of #8913.

Rationale for this change

This PR adds TreeNode::exists() API as a common pattern to check the presence of a matching node for a predicate in a tree.
This PR also solves the issue found by @alamb here: #9913 (comment). Since #9913 there is no need to call LogicalPlan::expressions() (that copies the plan's expressions) at certain places, but iterating over the expressions with LogicalPlan::apply_expressions() is enough.

What changes are included in this PR?

  • Adds the TreeNode::exists() API.
  • Replaces contains_outer_reference helper with a better solution.

Are these changes tested?

Yes, with existing UTs.

Are there any user-facing changes?

No.

@github-actions github-actions bot added logical-expr Logical plan and expressions optimizer Optimizer rules labels Apr 9, 2024
@peter-toth
Copy link
Contributor Author

@alamb, I think there are other places where LogicalPlan::expressions() can be replaced with the faster LogicalPlan::apply_expressions() but the change is non-trivial, so let's start with this smaller PR first.

@alamb alamb changed the title Introduce TreeNode::exists() API Introduce TreeNode::exists() API, avoid copying expressions Apr 9, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @peter-toth -- this looks great! (and thank you for keeping the PRs small).

I have some comment suggestions, but I also think we could add them as a follow on PR too


fn contains_outer_reference(inner_plan: &LogicalPlan) -> bool {
inner_plan
.expressions()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ for removing this copy 🚀

}
}
_ if plan.expressions().iter().any(|expr| expr.contains_outer()) => {
_ if plan.contains_outer_reference() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And another copy gone!

@alamb alamb changed the title Introduce TreeNode::exists() API, avoid copying expressions Introduce TreeNode::exists() API, avoid copying expressions Apr 9, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the liberty of adding the doc comments directly to this PR. Thanks again @peter-toth

@alamb alamb merged commit 57b2656 into apache:main Apr 9, 2024
@peter-toth
Copy link
Contributor Author

I took the liberty of adding the doc comments directly to this PR. Thanks again @peter-toth

Thanks @alamb for the doc comments and the review! Actually, I wanted to ask you to do that as I'm travelling this week and have limited access to github.

@alamb
Copy link
Contributor

alamb commented Apr 9, 2024

Thank you @peter-toth for all your help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions optimizer Optimizer rules

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants