To avoid allocations, we should use `PtrArray` from StrideArrays, but this is currently blocked by https://github.com/JuliaSIMD/StrideArrays.jl/issues/62.