-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support StringView for binary operators like ~
, !~
, etc
#12180
Comments
~
, !~
, etc~
, !~
, etc
Looks like #12168 also might be a starting point to solve this problem. I also think about solution 2 when handling that PR, and IMO solution 2 is the better option. I will take this one.
|
take |
I agree solution 2 is better -- it just will take longer as it needs two coordinated PRs. One thing we have done in the past is do the initial implementation in DataFusion, and then file a ticket / port the code upstream to arrow-rs. Once the code is release in arrow-rs and available in Datafusion we remove the copy in DataFusion |
I think this ticket is done now (though there is some follow up work to improve performance / port the code upstream) Thanks again @tlm365 |
Is your feature request related to a problem or challenge?
Part of #11752
As we work to complete StringView support in DataFusion @2010YOUY01 noticed on #11752 (comment) that we don't currently support Regexp like binary operators https://datafusion.apache.org/user-guide/sql/operators.html#op-re-match for string view
Reproducer
Describe the solution you'd like
StringView should be supported for these operators (aka the query should run without error)
Describe alternatives you've considered
Here are the relevant operator names:
Here is the dispatch code:
datafusion/datafusion/physical-expr/src/expressions/binary.rs
Lines 621 to 632 in 0f96af5
It appears that the corresponding arrow-rs kernel does not yet have support for StringView
https://docs.rs/arrow-string/52.2.0/src/arrow_string/regexp.rs.html#307-311
So what I would suggest is:
Additional context
No response
The text was updated successfully, but these errors were encountered: