Skip to content

Sort with limit panics for the limit includes some but not all nulls, for large arrays #235

@mathiaspeters-sig

Description

@mathiaspeters-sig

Describe the bug
We are getting the following panic:
thread panicked at 'source slice length (139885) does not match destination slice length (239865)' at https://github.com/apache/arrow-rs/blob/master/arrow/src/compute/kernels/sort.rs#L516
when we are trying to sort a large array with a large limit that includes some but not all of the nulls. We are on commit 0d1697f619c1bff8f64b19df5d4ec66634eaecf9.

To Reproduce
We have tried to reproduce this with smaller datasets but have been unable to. The smallest dataset we have confirmed this to happen on had 1mil rows.

Expected behavior
Sorting should work normally without panicing.

Additional context
We plan to work on this tomorrow (CET).

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrowChanges to the arrow cratebug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions