Skip to content

Commit d1a0d99

Browse files
jhammanjhamman
authored
V3 reorg (#1809)
* move v2 code/tests to v2 directories * move zarr v3 code to root * add extra doc deps * import __version__ in root __init__
1 parent f7993db commit d1a0d99

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

92 files changed

+2055
-2121
lines changed

‎docs/api.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,6 @@ API reference
1313
api/codecs
1414
api/attrs
1515
api/sync
16-
api/v3
1716

1817
Indices and tables
1918
------------------

‎docs/api/attrs.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
The Attributes class (``zarr.attrs``)
2-
=====================================
3-
.. module:: zarr.attrs
1+
The Attributes class (``zarr.v2.attrs``)
2+
========================================
3+
.. module:: zarr.v2.attrs
44

55
.. autoclass:: Attributes
66

‎docs/api/codecs.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
Compressors and filters (``zarr.codecs``)
2-
=========================================
1+
Compressors and filters (``zarr.v2.codecs``)
2+
============================================
33
.. module:: zarr.codecs
44

55
This module contains compressor and filter classes for use with Zarr. Please note that this module
@@ -8,9 +8,9 @@ onwards, all codec classes have been moved to a separate package called Numcodec
88
packages (Zarr and Numcodecs_) are designed to be used together. For example, a Numcodecs_ codec
99
class can be used as a compressor for a Zarr array::
1010

11-
>>> import zarr
11+
>>> import zarr.v2
1212
>>> from numcodecs import Blosc
13-
>>> z = zarr.zeros(1000000, compressor=Blosc(cname='zstd', clevel=1, shuffle=Blosc.SHUFFLE))
13+
>>> z = zarr.v2.zeros(1000000, compressor=Blosc(cname='zstd', clevel=1, shuffle=Blosc.SHUFFLE))
1414

1515
Codec classes can also be used as filters. See the tutorial section on :ref:`tutorial_filters`
1616
for more information.

‎docs/api/convenience.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
Convenience functions (``zarr.convenience``)
2-
============================================
3-
.. automodule:: zarr.convenience
1+
Convenience functions (``zarr.v2.convenience``)
2+
===============================================
3+
.. automodule:: zarr.v2.convenience
44
.. autofunction:: open
55
.. autofunction:: save
66
.. autofunction:: load

‎docs/api/core.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
The Array class (``zarr.core``)
2-
===============================
1+
The Array class (``zarr.v2.core``)
2+
==================================
33

4-
.. automodapi:: zarr.core
4+
.. automodapi:: zarr.v2.core
55
:no-heading:

‎docs/api/creation.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
Array creation (``zarr.creation``)
2-
==================================
3-
.. module:: zarr.creation
1+
Array creation (``zarr.v2.creation``)
2+
=====================================
3+
.. module:: zarr.v2.creation
44
.. autofunction:: create
55
.. autofunction:: empty
66
.. autofunction:: zeros

‎docs/api/hierarchy.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
Groups (``zarr.hierarchy``)
2-
===========================
3-
.. module:: zarr.hierarchy
1+
Groups (``zarr.v2.hierarchy``)
2+
==============================
3+
.. module:: zarr.v2.hierarchy
44

55
.. autofunction:: group
66
.. autofunction:: open_group

‎docs/api/n5.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
N5 (``zarr.n5``)
2-
================
3-
.. automodule:: zarr.n5
1+
N5 (``zarr.v2.n5``)
2+
===================
3+
.. automodule:: zarr.v2.n5
44

55
.. autoclass:: N5Store

‎docs/api/storage.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
Storage (``zarr.storage``)
2-
==========================
3-
.. automodule:: zarr.storage
1+
Storage (``zarr.v2.storage``)
2+
=============================
3+
.. automodule:: zarr.v2.storage
44

55
.. autoclass:: MemoryStore
66
.. autoclass:: DirectoryStore

‎docs/api/sync.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
Synchronization (``zarr.sync``)
2-
===============================
3-
.. module:: zarr.sync
1+
Synchronization (``zarr.v2.sync``)
2+
==================================
3+
.. module:: zarr.v2.sync
44

55
.. autoclass:: ThreadSynchronizer
66
.. autoclass:: ProcessSynchronizer

‎docs/api/v3.rst

Lines changed: 0 additions & 77 deletions
This file was deleted.

‎pyproject.toml

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,10 @@ docs = [
5252
'pydata-sphinx-theme',
5353
'numpydoc',
5454
'numcodecs[msgpack]',
55+
"msgpack",
56+
"lmdb",
57+
"zstandard",
58+
"crc32c",
5559
]
5660
extra = [
5761
'attrs',
@@ -164,31 +168,31 @@ disallow_any_generics = true
164168

165169
[[tool.mypy.overrides]]
166170
module = [
167-
"zarr._storage.store",
168-
"zarr._storage.v3_storage_transformers",
169-
"zarr.v3.group",
170-
"zarr.core",
171-
"zarr.hierarchy",
172-
"zarr.indexing",
173-
"zarr.storage",
174-
"zarr.sync",
175-
"zarr.util",
171+
"zarr.v2._storage.store",
172+
"zarr.v2._storage.v3_storage_transformers",
173+
"zarr.group",
174+
"zarr.v2.core",
175+
"zarr.v2.hierarchy",
176+
"zarr.v2.indexing",
177+
"zarr.v2.storage",
178+
"zarr.v2.sync",
179+
"zarr.v2.util",
176180
"tests.*",
177181
]
178182
check_untyped_defs = false
179183

180184
[[tool.mypy.overrides]]
181185
module = [
182-
"zarr.v3.abc.codec",
183-
"zarr.v3.codecs.bytes",
184-
"zarr.v3.codecs.pipeline",
185-
"zarr.v3.codecs.sharding",
186-
"zarr.v3.codecs.transpose",
187-
"zarr.v3.array_v2",
188-
"zarr.v3.array",
189-
"zarr.v3.sync",
190-
"zarr.convenience",
191-
"zarr.meta",
186+
"zarr.abc.codec",
187+
"zarr.codecs.bytes",
188+
"zarr.codecs.pipeline",
189+
"zarr.codecs.sharding",
190+
"zarr.codecs.transpose",
191+
"zarr.array_v2",
192+
"zarr.array",
193+
"zarr.sync",
194+
"zarr.v2.convenience",
195+
"zarr.v2.meta",
192196
]
193197
disallow_any_generics = false
194198

‎src/zarr/__init__.py

Lines changed: 40 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,45 @@
1-
# flake8: noqa
2-
from zarr.codecs import *
3-
from zarr.convenience import (
4-
consolidate_metadata,
5-
copy,
6-
copy_all,
7-
copy_store,
8-
load,
9-
open,
10-
open_consolidated,
11-
save,
12-
save_array,
13-
save_group,
14-
tree,
15-
)
16-
from zarr.core import Array
17-
from zarr.creation import (
18-
array,
19-
create,
20-
empty,
21-
empty_like,
22-
full,
23-
full_like,
24-
ones,
25-
ones_like,
26-
open_array,
27-
open_like,
28-
zeros,
29-
zeros_like,
30-
)
31-
from zarr.errors import CopyError, MetadataError
32-
from zarr.hierarchy import Group, group, open_group
33-
from zarr.n5 import N5Store, N5FSStore
34-
from zarr.storage import (
35-
ABSStore,
36-
DBMStore,
37-
DictStore,
38-
DirectoryStore,
39-
KVStore,
40-
LMDBStore,
41-
LRUStoreCache,
42-
MemoryStore,
43-
MongoDBStore,
44-
NestedDirectoryStore,
45-
RedisStore,
46-
SQLiteStore,
47-
TempStore,
48-
ZipStore,
1+
from __future__ import annotations
2+
3+
from typing import Union
4+
5+
import zarr.codecs # noqa: F401
6+
from zarr.array import Array, AsyncArray # noqa: F401
7+
from zarr.array_v2 import ArrayV2
8+
from zarr.config import RuntimeConfiguration # noqa: F401
9+
from zarr.group import AsyncGroup, Group # noqa: F401
10+
from zarr.metadata import runtime_configuration # noqa: F401
11+
from zarr.store import ( # noqa: F401
12+
StoreLike,
13+
make_store_path,
4914
)
50-
from zarr.sync import ProcessSynchronizer, ThreadSynchronizer
15+
from zarr.sync import sync as _sync
5116
from zarr._version import version as __version__
5217

5318
# in case setuptools scm screw up and find version to be 0.0.0
5419
assert not __version__.startswith("0.0.0")
20+
21+
22+
async def open_auto_async(
23+
store: StoreLike,
24+
runtime_configuration_: RuntimeConfiguration = RuntimeConfiguration(),
25+
) -> Union[AsyncArray, AsyncGroup]:
26+
store_path = make_store_path(store)
27+
try:
28+
return await AsyncArray.open(store_path, runtime_configuration=runtime_configuration_)
29+
except KeyError:
30+
return await AsyncGroup.open(store_path, runtime_configuration=runtime_configuration_)
31+
32+
33+
def open_auto(
34+
store: StoreLike,
35+
runtime_configuration_: RuntimeConfiguration = RuntimeConfiguration(),
36+
) -> Union[Array, ArrayV2, Group]:
37+
object = _sync(
38+
open_auto_async(store, runtime_configuration_),
39+
runtime_configuration_.asyncio_loop,
40+
)
41+
if isinstance(object, AsyncArray):
42+
return Array(object)
43+
if isinstance(object, AsyncGroup):
44+
return Group(object)
45+
raise TypeError(f"Unexpected object type. Got {type(object)}.")

‎src/zarr/_storage/__init__.py renamed to ‎src/zarr/abc/__init__.py

File renamed without changes.

‎src/zarr/v3/abc/codec.py renamed to ‎src/zarr/abc/codec.py

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,17 @@
44
from typing import TYPE_CHECKING, Optional
55

66
import numpy as np
7-
from zarr.v3.abc.metadata import Metadata
7+
from zarr.abc.metadata import Metadata
88

9-
from zarr.v3.common import ArraySpec
10-
from zarr.v3.store import StorePath
9+
from zarr.common import ArraySpec
10+
from zarr.store import StorePath
1111

1212

1313
if TYPE_CHECKING:
1414
from typing_extensions import Self
15-
from zarr.v3.common import BytesLike, SliceSelection
16-
from zarr.v3.metadata import ArrayMetadata
17-
from zarr.v3.config import RuntimeConfiguration
15+
from zarr.common import BytesLike, SliceSelection
16+
from zarr.metadata import ArrayMetadata
17+
from zarr.config import RuntimeConfiguration
1818

1919

2020
class Codec(Metadata):

‎src/zarr/v3/abc/metadata.py renamed to ‎src/zarr/abc/metadata.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
from dataclasses import fields, dataclass
99

10-
from zarr.v3.common import JSON
10+
from zarr.common import JSON
1111

1212

1313
@dataclass(frozen=True)

‎src/zarr/v3/abc/store.py renamed to ‎src/zarr/abc/store.py

File renamed without changes.

0 commit comments

Comments
 (0)