-
-
Notifications
You must be signed in to change notification settings - Fork 327
zarr.array
from from an existing zarr.Array
#2622
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
zarr.array
from from an existing zarr.Array
#2622
Conversation
# Conflicts: # tests/test_array.py
Do we also want concurrency for different chunk sizes? |
That would be nice, if the chunk sizes are somewhat compatible, i.e. one is a multiple of the other. |
if you are trying to write |
one question to answer here is what "auto" means for chunks if the user passes in a chunked array, but they want to use We might want to use a separate value that means "copy the chunks this object already has", which is distinct from "generate some chunks using the chunking heuristics". maybe something like |
Good point! I like the idea of distinguishing between |
…reation-from-other-zarr # Conflicts: # src/zarr/core/array.py
# Conflicts: # src/zarr/api/asynchronous.py # src/zarr/api/synchronous.py # src/zarr/core/array.py # tests/test_indexing.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is ready. @d-v-b what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good, thanks @brokkoli71
* add creation from other zarr * remove duplicated tests * improve test * test_iter_grid for non-squares * concurrent streaming for equal chunk sizes * fix merge * fix mypy * fix mypy * fix test_iter_grid * extract to zarr.from_array * fix mypy * fix mypy * format * fix test_creation_from_other_zarr_format * distinguish between keep and auto for from_array arguments * partition concurrency along new_array chunks * fix mypy * improve test_creation_from_other_zarr_format * add typing in test * Update src/zarr/core/array.py Co-authored-by: Norman Rzepka <code@normanrz.com> * add from_array with npt.ArrayLike * add write_data argument * improve tests * improve docstrings and add examples * fix mypy and readthedocs * fix mypy and readthedocs * fix mypy and readthedocs * fix mypy and readthedocs * fix readthedocs ERROR: Unexpected indentation * add release notes * format docstring examples * add write_data attr to synchronous.create_array * `create_array` calls `from_array` calls `init_array` * document changes * fix serializer from_array v2 to v3 * fix mypy * improve codecov * fix mypy * from_array: copy zarr format on default * in ``from_array`` make all arguments except ``store`` keyword-only, to match ``create_array`` * in ``from_array`` default shards="keep" * redundant ``ChunkKeyEncoding | ChunkKeyEncodingLike`` * fix argument order in calls of `from_array` * fix numpydoc-validation * add docstring to store2 pytest fixture * extract `_parse_keep_array_attr` from `from_array` * extract `_parse_keep_array_attr` from `from_array` * correct _parse_keep_array_attr * fix merge * fix merge --------- Co-authored-by: Norman Rzepka <code@normanrz.com>
zarr.Array
#2410added concurrent streaming of source array into new array
TODO: