Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zarr v3 support #292

Closed
wants to merge 7 commits into from
Closed

Zarr v3 support #292

wants to merge 7 commits into from

Conversation

jhamman
Copy link
Contributor

@jhamman jhamman commented Feb 2, 2023

This PR experiments with adding Zarr v3 support to Kerchunk.

In most cases, this simply requires passing the zarr_version parameter to the zarr.group constructor. I've decided to leave multizarr out of this PR for now. Its going to take more work and I'd like to avoid the larger refactor that may be needed for now.

If you are reviewing this, I suggest starting with the hdf5 translator and the Python 3.10 GitHub action.

towards #235
second try after #237

@martindurant
Copy link
Member

Am I right to think that V3 reference sets will currently fail in MultiZarrToZarr? Do you have an estimate of how complex it would be to support?

@jhamman
Copy link
Contributor Author

jhamman commented Apr 6, 2023

Am I right to think that V3 reference sets will currently fail in MultiZarrToZarr? Do you have an estimate of how complex it would be to support?

Yes, I started down the path of reworking this but I found it to be too complicated. There are just so many places to special case key names that I wasn't sure it was worth the effort.

@martindurant
Copy link
Member

Yes, I started down the path of reworking this but I found it to be too complicated. There are just so many places to special case key names that I wasn't sure it was worth the effort.

I wonder if it might be simpler to create a v2->v3 converter. Of course, in the long run we want v3 to become the norm.

@jhamman
Copy link
Contributor Author

jhamman commented Sep 13, 2024

Closing this as a) super out of date and b) not in line with the v3 spec. I would encourage those following this PR to revisit the v2/v3 conversation after zarr-python 3 is released.

@jhamman jhamman closed this Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants