Skip to content

Implement N5Store as a transformation layer over other stores #9

@alimanfoo

Description

@alimanfoo

The current work on an N5Store in zarr-developers/zarr-python#309 implements a storage class that does two things. First, it implements a transformation from the keys and values used in the zarr protocol into keys and values used in the N5 protocol. It then also implements storage of keys and values as files on a file system.

It would be possible to separate these concerns, such that the N5Store just implemented the protocol transformation, and then delegated the actual storage or retrieval of N5 keys and values from an inner store. This would mean that the N5Store class could be used with file systems or cloud stores or any form of key/value storage.

E.g., what is currently now:

store = zarr.N5Store('/path/to/store.n5')

...could become a shorthand for:

store = zarr.N5Store(zarr.DirectoryStore('/path/to/store.n5'))

...and other inner stores could be used, e.g.:

from gcsfs import GCSMap
store = zarr.N5Store(GCSMap('my-bucket', ...))

Internally, to make this possible, the N5Store class would replace all the logic for opening, reading or writing files with calls to inner.__getitem__(key) and inner.__setitem__(key, value) where inner is the inner store wrapped by the N5Store class.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions