Skip to content

Arrow IPC projection support #1338

@Dandandan

Description

@Dandandan

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The FileReader currently doesn't support projection to skip reading/decoding certain columns.
This could be used by engines lole DataFusion to make the Arrow file reader more efficient.

Describe the solution you'd like
Support an optional Vec<usize> parameter to configure the column indexes in the reader.

Describe alternatives you've considered

Additional context
See support in arrow2
https://docs.rs/arrow2/latest/arrow2/io/ipc/read/struct.FileReader.html

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changelog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions