-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Closed
Labels
performanceMake DataFusion fasterMake DataFusion faster
Description
When implementing functions, ColumnarValue::into_array / ScalarValue::to_array / ScalarValue::to_array_of_size are useful to reduce code complexity and denominate input arguments in terms of an array (either full-sized or 1-sized).
The code to handle scalars of structural types is
datafusion/datafusion/common/src/scalar/mod.rs
Lines 2341 to 2354 in 049ca15
| ScalarValue::List(arr) => { | |
| Self::list_to_array_of_size(arr.as_ref() as &dyn Array, size)? | |
| } | |
| ScalarValue::LargeList(arr) => { | |
| Self::list_to_array_of_size(arr.as_ref() as &dyn Array, size)? | |
| } | |
| ScalarValue::FixedSizeList(arr) => { | |
| Self::list_to_array_of_size(arr.as_ref() as &dyn Array, size)? | |
| } | |
| ScalarValue::Struct(arr) => { | |
| Self::list_to_array_of_size(arr.as_ref() as &dyn Array, size)? | |
| } | |
| ScalarValue::Map(arr) => { | |
| Self::list_to_array_of_size(arr.as_ref() as &dyn Array, size)? |
it's not optimized for the common case of input array (the scalar) being of length 1 and requested length also being 1.
In such case it could probably be reduced to Arc::clone
tlm365 and comphead
Metadata
Metadata
Assignees
Labels
performanceMake DataFusion fasterMake DataFusion faster