Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: Binder should not simplify isNull and notNull when their reference … #7346

Closed

Conversation

zhongyujiang
Copy link
Contributor

…fields are required but nested in optional parents.

Currently, Binder will simpify isNull and notNull to alwaysFalse and alwaysTrue respectively, when the expressions are to be bound to required fields. However a required field can still be null in the row when it is nested in an optional parent field. Therefore, this causes data to be incorrectly skipped when querying for rows where their nested field(required sub field nested in optional parent) is null.

This changes the Binder to only simplify nullability predicates when their target field and all its parents are all required, to fix this issue.


checkOnlyIcebergFilters(
"point.x IS NULL" /* query predicate */,
"point.x IS NULL" /* Iceberg scan filters */,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This query returns no rows without this PR.

@aokolnychyi
Copy link
Contributor

Seems like a good catch. I'll try to review the actual fix later this week unless someone gets there earlier.

cc @nastra @rdblue @szehon-ho @flyrain @amogh-jahagirdar @jackye1995

@zhongyujiang
Copy link
Contributor Author

Hey @aokolnychyi , do you have time to review this?

@zhongyujiang zhongyujiang force-pushed the fix-binder-simplifiction branch from 527a203 to e6989ae Compare September 8, 2023 12:25
@zhongyujiang zhongyujiang changed the title API: Binder should not simplify isNull and notNull when their target … API: Binder should not simplify isNull and notNull when their reference … Sep 8, 2023
Copy link

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions bot added the stale label Aug 29, 2024
Copy link

github-actions bot commented Sep 5, 2024

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions bot closed this Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants