Allow specialized implementations to produce hints for the array adapter #3765
Conversation
There was a problem hiding this comment.
Makes sense to me -- thank you @isidentical
I wonder if you have any benchmark results that show this pattern improving performance?
| /// - (default) `false`: indicates the argument needs to be padded if it is scalar | ||
| /// - `true`: indicates the argument can be converted to an array of length 1 | ||
| /// | ||
| pub(crate) fn make_scalar_function_with_hints<F>( |
There was a problem hiding this comment.
The fact that this is pub(crate) is good given its specialized nature (and potential possibility of change)
Yep, I was playing with the idea of using the existing null buffers for Also a subsequent optimization (using the existing null buffers) was unnoticeable before this change, but now we can clearly see the benefit of it since the actual time is now spent on result array construction inside |
isidentical
force-pushed
the
gh-3762
branch
2 times, most recently
from
1e858c5
to
aa9667a
Compare
| (_, true, true, true) => Ok(make_scalar_function_with_hints( | ||
| _regexp_replace_static_pattern_replace::<T>, | ||
| vec
|
Thanks again @isidentical |
|
Benchmark runs are scheduled for baseline = 8db6af2 and contender = f132259. f132259 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #3762.
Rationale for this change
More details on #3762, but simply this allows us to instruct array/scalar function adapter to skip padding some arguments since the specialized version of functions don't need them.
What changes are included in this PR?
A new
make_scalar_function_with_hintsAPI (not public) that can take a list of hints and then map them to corresponding arguments and use it to decide whether they need padding or not.This PR also provides usage of the new hints API, which in turn can speed-up
regex_replaceup to 1.3x.Are there any user-facing changes?
No, this is a new API / optimization that can provide up to %25 speed-up.