Skip to content

Commit aff78ce

Browse files
committed
Make partial decompress opt-in, extend whatsnew.
1 parent ce9182c commit aff78ce

File tree

3 files changed

+30
-10
lines changed

3 files changed

+30
-10
lines changed

docs/release.rst

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,14 +31,16 @@ See `this link <https://github.com/zarr-developers/zarr-python/milestone/11?clos
3131
merged PR tagged with the 2.6 milestone.
3232

3333
* Add ability to partially read and decompress arrays, see :issue:`667`. IT is
34-
only available to chunks fored via fs-spec and using bloc as a compressor.
34+
only available to chunks stored using fs-spec and using bloc as a compressor.
3535

3636
For certain analysis case when only a small portion of chunks is needed it can
3737
be advantageous to only access and decompress part of the chunks. Doing
3838
partial read and decompression add high latency to many of the operation so
3939
should be used only when the subset of the data is small compared to the full
40-
chunks and is stored contiguously (last dimensions for C layout, firsts for
41-
F). Set ``partial_decompress=True`` when creating an ``Array``
40+
chunks and is stored contiguously (that is to say either last dimensions for C
41+
layout, firsts for F). Pass ``partial_decompress=True`` as argument when
42+
creating an ``Array``, or when using ``open_array``. No option exists yet to
43+
apply partial read and decompress on a per-operation basis.
4244

4345
2.5.0
4446
-----

zarr/core.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,10 +71,12 @@ class Array:
7171
operations. If False, user attributes are reloaded from the store prior
7272
to all attribute read operations.
7373
partial_decompress : bool, optional
74-
If True and while the chunk_store is a FSStore and the compreesion used
74+
If True and while the chunk_store is a FSStore and the compresion used
7575
is Blosc, when getting data from the array chunks will be partially
7676
read and decompressed when possible.
7777
78+
.. versionadded:: 2.7
79+
7880
Attributes
7981
----------
8082
store
@@ -134,7 +136,7 @@ def __init__(
134136
synchronizer=None,
135137
cache_metadata=True,
136138
cache_attrs=True,
137-
partial_decompress=True,
139+
partial_decompress=False,
138140
):
139141
# N.B., expect at this point store is fully initialized with all
140142
# configuration metadata fully specified and normalized

zarr/creation.py

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -362,11 +362,26 @@ def array(data, **kwargs):
362362
return z
363363

364364

365-
def open_array(store=None, mode='a', shape=None, chunks=True, dtype=None,
366-
compressor='default', fill_value=0, order='C', synchronizer=None,
367-
filters=None, cache_metadata=True, cache_attrs=True, path=None,
368-
object_codec=None, chunk_store=None, storage_options=None,
369-
**kwargs):
365+
def open_array(
366+
store=None,
367+
mode="a",
368+
shape=None,
369+
chunks=True,
370+
dtype=None,
371+
compressor="default",
372+
fill_value=0,
373+
order="C",
374+
synchronizer=None,
375+
filters=None,
376+
cache_metadata=True,
377+
cache_attrs=True,
378+
path=None,
379+
object_codec=None,
380+
chunk_store=None,
381+
storage_options=None,
382+
partial_decompress=False,
383+
**kwargs
384+
):
370385
"""Open an array using file-mode-like semantics.
371386
372387
Parameters
@@ -415,6 +430,7 @@ def open_array(store=None, mode='a', shape=None, chunks=True, dtype=None,
415430
storage_options : dict
416431
If using an fsspec URL to create the store, these will be passed to
417432
the backend implementation. Ignored otherwise.
433+
partial_decompress:
418434
419435
Returns
420436
-------

0 commit comments

Comments
 (0)