-
-
Notifications
You must be signed in to change notification settings - Fork 331
Add API reference for V3 Implementation in the docs #1345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
joshmoore
merged 8 commits into
zarr-developers:main
from
sanketverma1704:v3_doc_update
Mar 17, 2023
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
91b4e0f
Add API reference for V3 Implementation in the docs
sanketverma1704 09d8640
Minor fix
sanketverma1704 23431a4
Minor fix
sanketverma1704 cd9a75f
Minor indentation fix
sanketverma1704 98a9ad0
Update docs/api/v3.rst
sanketverma1704 4c1d151
Update docs/api/v3.rst
sanketverma1704 d8a4131
Update docs/api/v3.rst
sanketverma1704 b17b17a
Fix broken links
sanketverma1704 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,6 +13,7 @@ API reference | |
api/codecs | ||
api/attrs | ||
api/sync | ||
api/v3 | ||
|
||
Indices and tables | ||
------------------ | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,77 @@ | ||
V3 Specification Implementation(``zarr._storage.v3``) | ||
===================================================== | ||
|
||
This module contains the implementation of the `Zarr V3 Specification <https://zarr-specs.readthedocs.io/en/latest/v3/core/v3.0.html>`_. | ||
|
||
.. warning:: | ||
Since Zarr Python 2.12 release, this module provides experimental infrastructure for reading and | ||
writing the upcoming V3 spec of the Zarr format. Users wishing to prepare for the migration can set | ||
the environment variable ``ZARR_V3_EXPERIMENTAL_API=1`` to begin experimenting, however data | ||
written with this API should be expected to become stale, as the implementation will still change. | ||
|
||
The new ``zarr._store.v3`` package has the necessary classes and functions for evaluating Zarr V3. | ||
Since the design is not finalised, the classes and functions are not automatically imported into | ||
the regular Zarr namespace. | ||
|
||
Code snippet for creating Zarr V3 arrays:: | ||
|
||
>>> import zarr | ||
>>> z = zarr.create((10000, 10000), | ||
>>> chunks=(100, 100), | ||
>>> dtype='f8', | ||
>>> compressor='default', | ||
>>> path='path-where-you-want-zarr-v3-array', | ||
>>> zarr_version=3) | ||
|
||
Further, you can use `z.info` to see details about the array you just created:: | ||
|
||
>>> z.info | ||
Name : path-where-you-want-zarr-v3-array | ||
Type : zarr.core.Array | ||
Data type : float64 | ||
Shape : (10000, 10000) | ||
Chunk shape : (100, 100) | ||
Order : C | ||
Read-only : False | ||
Compressor : Blosc(cname='lz4', clevel=5, shuffle=SHUFFLE, blocksize=0) | ||
Store type : zarr._storage.v3.KVStoreV3 | ||
No. bytes : 800000000 (762.9M) | ||
No. bytes stored : 557 | ||
Storage ratio : 1436265.7 | ||
Chunks initialized : 0/10000 | ||
|
||
You can also check ``Store type`` here (which indicates Zarr V3). | ||
|
||
.. module:: zarr._storage.v3 | ||
|
||
.. autoclass:: RmdirV3 | ||
.. autoclass:: KVStoreV3 | ||
.. autoclass:: FSStoreV3 | ||
.. autoclass:: MemoryStoreV3 | ||
.. autoclass:: DirectoryStoreV3 | ||
.. autoclass:: ZipStoreV3 | ||
.. autoclass:: RedisStoreV3 | ||
.. autoclass:: MongoDBStoreV3 | ||
.. autoclass:: DBMStoreV3 | ||
.. autoclass:: LMDBStoreV3 | ||
.. autoclass:: SQLiteStoreV3 | ||
.. autoclass:: LRUStoreCacheV3 | ||
.. autoclass:: ConsolidatedMetadataStoreV3 | ||
|
||
In v3 `storage transformers <https://zarr-specs.readthedocs.io/en/latest/v3/array-storage-transformers/sharding/v1.0.html>`_ | ||
can be set via ``zarr.create(…, storage_transformers=[…])``. | ||
The experimental sharding storage transformer can be tested by setting | ||
the environment variable ``ZARR_V3_SHARDING=1``. Data written with this flag | ||
enabled should be expected to become stale until | ||
`ZEP 2 <https://zarr.dev/zeps/draft/ZEP0002.html>`_ is approved | ||
and fully implemented. | ||
|
||
.. module:: zarr._storage.v3_storage_transformers | ||
|
||
.. autoclass:: ShardingStorageTransformer | ||
|
||
The abstract base class for storage transformers is | ||
|
||
.. module:: zarr._storage.store | ||
|
||
.. autoclass:: StorageTransformer |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.