Skip to content

Inefficient reading of slices of a Dataset #6

@jjhelmus

Description

@jjhelmus

When reading data from a Dataset, pyfive currently loads all chunks into memory before slicing the requested data. This behavior is inefficient when only a small region of the data is required which could be extracted from a small number or even a single chunk.

The code used for slicing dask arrays may be helpful for determining which chunks need to be read for the given slice.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions