-
Couldn't load subscription status.
- Fork 1k
Closed
Labels
Description
Describe the bug
Noticed whilst working on #1225, calling MutableArrayData::extend_nulls doesn't update the null bitmask. This means the final array may have a shifted or completely invalid bitmask.
#1225 changed it so that if MutableArrayData is created with nulls disabled, it panics if you try to append nulls. Unfortunately it would appear this is being exploited by ArrowArrayReader to avoid computing a bitmask.
To Reproduce
#[test]
fn test_nulls() {
let ints: ArrayRef = Arc::new(Int32Array::from(vec![1]));
let mut data = MutableArrayData::new(vec![ints.data()], true, 3);
data.extend_nulls(9);
let data = data.freeze();
assert_eq!(data.len(), 9);
assert_eq!(data.null_buffer().unwrap().len(), 2);
}
Expected behavior
Appending nulls should update the null bitmask, and it should panic if MutableArrayData is created without null support. Fixing this is likely blocked on #1197 as it relies on this behaviour