Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharding in zarr v2 #2346

Open
d-v-b opened this issue Oct 12, 2024 · 0 comments
Open

Sharding in zarr v2 #2346

d-v-b opened this issue Oct 12, 2024 · 0 comments

Comments

@d-v-b
Copy link
Contributor

d-v-b commented Oct 12, 2024

Since sharding is implemented as a codec, and because the zarr v2 spec takes a permissive stance to codecs, the sharding codec can be used for zarr v2 arrays, provided a program exists that can interpret that codec, such as any zarr v3 implementation. I can think of a variety of use cases for this (e.g., all use cases imagined for sharding in zarr v3), but the most simple one might be for people with v2 data who want sharding but don't want to change anything else about their workflow.

FWIW I haven't tested what the v3 branch of zarr-python would do with a v2 array with a sharding codec, but IMO failing to open it would be a bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant