Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify small InList expressions #4089

Closed
alamb opened this issue Nov 2, 2022 · 0 comments · Fixed by #4090
Closed

Simplify small InList expressions #4089

alamb opened this issue Nov 2, 2022 · 0 comments · Fixed by #4090
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@alamb
Copy link
Contributor

alamb commented Nov 2, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
You can write a query like

select ... from foo where id in (4)

And that sql is oftentimes made by tools that handle some number of ids;

We have a specialized InList implementation (e.g. see #4057) but for single values it is still faster to use a standard equality predicate

Describe the solution you'd like

As mentioned by @jackwener and @Dandandan in #4057 (comment)_ we should rewrite inlist with a few elements.

We should definitely simplify <left> IN (<expr>) to <left> = <expr> as that will be better in all cases.

Describe alternatives you've considered

We could potentially also rewrite <left> IN (<expr>, <expr2>, .. <exprN>) to <left> = <expr> OR <left> = <expr2> OR .. <left> = <exprN>

However, at some point the InList expression is faster to evaluate, and that break even point depends on the cost to evaluate <left> . Thus I suggest we only rewrite for single value IN lists

Additional context
This is a good first issue because there are several examples of the code and tests to follow

You can find simplify rules here: https://github.com/apache/arrow-datafusion/blob/10e64dc013ba210ab1f6c2a3c02c66aef4a0e802/datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs#L329-L339

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant