-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Open
Labels
enhancementAny new improvement worthy of a entry in the changelogAny new improvement worthy of a entry in the changelog
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
While we appear to support listviews in interleave via the fallback, we can consider adding a native implementation if it is faster. We already do so for lists:
arrow-rs/arrow-select/src/interleave.rs
Lines 96 to 112 in 576a063
| downcast_primitive! { | |
| data_type => (primitive_helper, values, indices, data_type), | |
| DataType::Utf8 => interleave_bytes::<Utf8Type>(values, indices), | |
| DataType::LargeUtf8 => interleave_bytes::<LargeUtf8Type>(values, indices), | |
| DataType::Binary => interleave_bytes::<BinaryType>(values, indices), | |
| DataType::LargeBinary => interleave_bytes::<LargeBinaryType>(values, indices), | |
| DataType::BinaryView => interleave_views::<BinaryViewType>(values, indices), | |
| DataType::Utf8View => interleave_views::<StringViewType>(values, indices), | |
| DataType::Dictionary(k, _) => downcast_integer! { | |
| k.as_ref() => (dict_helper, values, indices), | |
| _ => unreachable!("illegal dictionary key type {k}") | |
| }, | |
| DataType::Struct(fields) => interleave_struct(fields, values, indices), | |
| DataType::List(field) => interleave_list::<i32>(values, indices, field), | |
| DataType::LargeList(field) => interleave_list::<i64>(values, indices, field), | |
| _ => interleave_fallback(values, indices) | |
| } |
Describe the solution you'd like
Add native implementation of listvews to interleave without relying on fallback; would be good to see benchmarks to prove this is a performance boost worth doing.
Describe alternatives you've considered
Additional context
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementAny new improvement worthy of a entry in the changelogAny new improvement worthy of a entry in the changelog