Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expand LIKE simplification: cover NULL pattern/expression and constant #13260

Merged
merged 4 commits into from
Nov 8, 2024
Merged

Expand LIKE simplification: cover NULL pattern/expression and constant #13260

merged 4 commits into from
Nov 8, 2024
+257 −97

Conversation

findepi
Copy link
Member

@findepi findepi commented Nov 5, 2024 •
edited
Loading

@github-actions github-actions bot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Nov 5, 2024
@findepi findepi force-pushed the findepi/expand-like-simplification-e96eca branch from c6285bf to be26107 Compare November 5, 2024 13:39
@findepi
Copy link
Member Author

findepi commented Nov 5, 2024

currently depends on #13259

@findepi findepi mentioned this pull request Nov 5, 2024
Open
@findepi
Copy link
Member Author

findepi commented Nov 5, 2024

CI is green, so ready to review
but this will conflict with @adriangb's #13061, because this changes code structure a little bit
conflict resolution will be trivial though.
@adriangb if you want, i can do this, adding your changes to this PR

@findepi
Copy link
Member Author

findepi commented Nov 5, 2024

seemed easy enough, done.

@findepi findepi force-pushed the findepi/expand-like-simplification-e96eca branch from 88d6df1 to 6c57af7 Compare November 5, 2024 15:14
@adriangb
Copy link
Contributor

adriangb commented Nov 5, 2024

I’m happy with my changes being included in this PR :)

@findepi findepi marked this pull request as draft November 5, 2024 15:21
@findepi
Copy link
Member Author

findepi commented Nov 5, 2024

draft - to be rebased after #13259 lands

still ready to review
cc @crepererum @goldmedal

goldmedal
goldmedal reviewed Nov 5, 2024
View reviewed changes
Copy link
Contributor

@goldmedal goldmedal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just roughly review now. Overall looks to me. I will check the test cases tomorrow.

datafusion/optimizer/Cargo.toml Outdated Show resolved Hide resolved
@adriangb adriangb mentioned this pull request Nov 5, 2024
Closed
@goldmedal goldmedal mentioned this pull request Nov 6, 2024
Merged
@goldmedal
Copy link
Contributor

draft - to be rebased after #13259 lands

still ready to review cc @crepererum @goldmedal

#13259 has been merged. 👍

@findepi findepi force-pushed the findepi/expand-like-simplification-e96eca branch from e4eae46 to 520ad2b Compare November 6, 2024 07:31
@findepi findepi marked this pull request as ready for review November 6, 2024 07:31
@findepi
Copy link
Member Author

findepi commented Nov 6, 2024

@goldmedal rebased, thanks!

alamb
alamb reviewed Nov 6, 2024
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @findepi and @goldmedal

This looks like a great change to me except for the handling of %% which I am not sure about. Otherwise 👍

datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs
@@ -2987,41 +3051,41 @@ mod tests {
})
}

fn like(expr: Expr, pattern: &str) -> Expr {
fn like(expr: Expr, pattern: impl Into<Expr>) -> Expr {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could use Expr::like https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.Expr.html#method.like and similar here instead of these functions

This is likely left over from when the Expr API was less expressive

datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs Show resolved Hide resolved
datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs Show resolved Hide resolved
datafusion/sqllogictest/test_files/string/string_view.slt Outdated Show resolved Hide resolved
@alamb alamb changed the title Expand LIKE simplification Expand LIKE simplification: cover NULL pattern/expression and constant Nov 6, 2024
@crepererum
Copy link
Contributor

since there are already two reviewers on it, I'll gonna skip this PR. However if you need my input, ping me.

alamb
alamb approved these changes Nov 6, 2024
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @findepi and @goldmedal

I had some small testing suggestions but I think we can add that coverage as a follow on PR as well

@findepi findepi mentioned this pull request Nov 7, 2024
Merged
@findepi
Copy link
Member Author

findepi commented Nov 7, 2024

FYI, when testing LIKE patterns with potential escapes, beware of difference between sqllogictests and CLI when it comes to backslash (like implicit escape) -- #13286

@alamb
Copy link
Contributor

alamb commented Nov 7, 2024 •
edited
Loading

Looks like this branch has some conflicts now to resolve but then I think it will be ready to go

findepi and others added 2 commits November 8, 2024 15:57
@findepi
Expand LIKE simplification
96a6cd2 
- cover expression known not to be null
- cover NULL pattern
- cover repeated '%%' in pattern
@adriangb @findepi
Simplify EXPR LIKE 'constant' to expr = 'constant'
5a03823
@findepi
Copy link
Member Author

findepi commented Nov 8, 2024

The conflicts are because #13288 is now merged. Let me rebase.

@findepi findepi force-pushed the findepi/expand-like-simplification-e96eca branch from e05417a to 5a03823 Compare November 8, 2024 14:58
@findepi
Copy link
Member Author

findepi commented Nov 8, 2024

(just rebased)

@findepi
Copy link
Member Author

findepi commented Nov 8, 2024

Applied comments and fixed handling of implicit escape.

alamb
alamb approved these changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@alamb
Copy link
Contributor

alamb commented Nov 8, 2024

Thanks @findepi @adriangb and @goldmedal for this PR and the reviews

@alamb alamb merged commit 667b302 into apache:main Nov 8, 2024
27 checks passed
@findepi findepi deleted the findepi/expand-like-simplification-e96eca branch November 8, 2024 18:54
@findepi findepi mentioned this pull request Nov 8, 2024
Open
jayzhan211 pushed a commit to jayzhan211/datafusion that referenced this pull request Nov 12, 2024
@findepi @adriangb @jayzhan211
Expand LIKE simplification: cover NULL pattern/expression and const…
9278483 
…ant (apache#13260)

* Expand LIKE simplification

- cover expression known not to be null
- cover NULL pattern
- cover repeated '%%' in pattern

* Simplify `EXPR LIKE 'constant'` to `expr = 'constant'`

* Correctness and style fixes

* fix typo

---------

Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@goldmedal goldmedal goldmedal left review comments

@alamb alamb alamb approved these changes

Assignees
No one assigned
Labels
optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@findepi @adriangb @goldmedal @crepererum @alamb