Fix incorrect ... LIKE '%' simplification with NULLs#13259
Merged
goldmedal merged 3 commits intoapache:mainfrom Nov 6, 2024
Merged
Fix incorrect ... LIKE '%' simplification with NULLs#13259goldmedal merged 3 commits intoapache:mainfrom
... LIKE '%' simplification with NULLs#13259goldmedal merged 3 commits intoapache:mainfrom
Conversation
github-actions
bot
added
optimizer
This was referenced Nov 5, 2024
Member
Author
|
this will conflict with #13061 cc @adriangb |
findepi
force-pushed
the
findepi/fix-incorrect-like-simplification-2c0bcb
branch
from
6c063d9 to
eedd384
Compare
findepi
commented
Nov 5, 2024
| Expr::Literal(ScalarValue::Utf8View(Some(pattern_str))) if pattern_str == "%" | ||
| ) => | ||
| { | ||
| Transformed::yes(lit(!negated)) |
`expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known.
findepi
force-pushed
the
findepi/fix-incorrect-like-simplification-2c0bcb
branch
from
eedd384 to
3085835
Compare
goldmedal
approved these changes
Nov 5, 2024
datafusion/sqllogictest/test_files/string/string_query.slt.part
Outdated
Show resolved
Hide resolved
alamb
changed the title
... LIKE '%' simplification... LIKE '%' simplification with NULLs
Contributor
|
Thanks, @findepi and @crepererum for reviewing. |
findepi
added a commit
to sdf-labs/datafusion
that referenced
this pull request
Nov 6, 2024
* Fix incorrect `... LIKE '%'` simplification `expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known. * fixup! Fix incorrect `... LIKE '%'` simplification * fix tests (re review comments)
findepi
added a commit
to sdf-labs/datafusion
that referenced
this pull request
Nov 6, 2024
…che#13259) * Fix incorrect `... LIKE '%'` simplification `expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known. * fixup! Fix incorrect `... LIKE '%'` simplification * fix tests (re review comments)
findepi
added a commit
to sdf-labs/datafusion
that referenced
this pull request
Nov 6, 2024
…che#13259) * Fix incorrect `... LIKE '%'` simplification `expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known. * fixup! Fix incorrect `... LIKE '%'` simplification * fix tests (re review comments)
findepi
added a commit
to sdf-labs/datafusion
that referenced
this pull request
Nov 6, 2024
…che#13259) * Fix incorrect `... LIKE '%'` simplification `expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known. * fixup! Fix incorrect `... LIKE '%'` simplification * fix tests (re review comments)
findepi
added a commit
to sdf-labs/datafusion
that referenced
this pull request
Nov 14, 2024
* Fix incorrect `... LIKE '%'` simplification `expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known. * fixup! Fix incorrect `... LIKE '%'` simplification * fix tests (re review comments)
felipecrv
pushed a commit
to sdf-labs/datafusion
that referenced
this pull request
Feb 3, 2025
* Fix incorrect `... LIKE '%'` simplification `expr LIKE '%'` was previously simplified to `true`, but the expression returns `NULL` when `expr` is null. The conversion was conditional on `!is_null(expr)` which means "is not always true, i.e. is not a null literal". This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite `utf8_view LIKE '%'` returning incorrect results at execution time (tracked by apache#12637). I.e. the simplification masks the bug for cases where pattern is statically known. * fixup! Fix incorrect `... LIKE '%'` simplification * fix tests (re review comments)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
expr LIKE '%'was previously simplified totrue, but the expression returnsNULLwhenexpris null.The conversion was conditional on
!is_null(expr)which means "is not always true, i.e. is not a null literal".This commit adds correct simplification logic. It additionally expands the rule coverage to include string view (Utf8View) and large string (LargeUtf8). This allows writing shared test cases even despite
utf8_view LIKE '%'returning incorrect results at execution time (tracked by #12637). I.e. the simplification masks the bug for cases where pattern is statically known.Which issue does this PR close?
Rationale for this change
fix correctness
What changes are included in this PR?
correctness fix for
... LIKE '%'simplificationAre these changes tested?
yes
Are there any user-facing changes?
bug fix