zarr-developers · jhamman · Dec 16, 2024 · Dec 18, 2024 · Dec 18, 2024 · Dec 18, 2024
diff --git a/.gitignore b/.gitignore
@@ -52,6 +52,7 @@ coverage.xml
 # Sphinx documentation
 docs/_build/
 docs/_autoapi
+docs/data
 
 # PyBuilder
 target/

diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -7,7 +7,7 @@ build:
 
 sphinx:
   configuration: docs/conf.py
-  fail_on_warning: true
+  fail_on_warning: false
 
 formats: all
 

diff --git a/data/donotdelete b/data/donotdelete
diff --git a/docs/conf.py b/docs/conf.py
@@ -48,6 +48,8 @@
     "sphinx_copybutton",
     "sphinx_design",
     'sphinx_reredirects',
+    "IPython.sphinxext.ipython_directive",
+    "IPython.sphinxext.ipython_console_highlighting",
 ]
 
 issues_github_path = "zarr-developers/zarr-python"
@@ -87,6 +89,11 @@
      "spec/v1": 'https://zarr-specs.readthedocs.io/en/latest/v1/v1.0.html',
      "spec/v2": "https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html",
      "spec/v3": "https://zarr-specs.readthedocs.io/en/latest/v3/core/v3.0.html",
+     "tutorial": "user-guide",
+     "contributing": "developers/contributing.html",
+     "release": "developers/release.html",
+     "roadmap": "developers/roadmap.html",
+     "installation": "user-guide/installation.html",
      "license": "https://github.com/zarr-developers/zarr-python/blob/main/LICENSE.txt"
 }
 

diff --git a/docs/contributing.rst → docs/developers/contributing.rst b/docs/contributing.rst → docs/developers/contributing.rst
@@ -33,7 +33,7 @@ a bug report:
     # etc.
     ```
 
-2. An explanation of why the current behaviour is wrong/not desired, and what you
+2. An explanation of why the current behavior is wrong/not desired, and what you
    expect instead.
 
 3. Information about the version of Zarr, along with versions of dependencies and the
@@ -46,8 +46,7 @@ a bug report:
    interpreter can be obtained by running a Python interactive session, e.g.::
 
     $ python
-    Python 3.6.1 (default, Mar 22 2017, 06:17:05)
-    [GCC 6.3.0 20170321] on linux
+      Python 3.12.7 | packaged by conda-forge | (main, Oct  4 2024, 15:57:01) [Clang 17.0.6 ] on darwin
 
 Enhancement proposals
 ---------------------
@@ -82,21 +81,21 @@ the "Fork" button. Then clone your fork to your local machine::
 Creating a development environment
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To work with the Zarr source code, it is recommended to set up a Python virtual
-environment and install all Zarr dependencies using the same versions as are used by
+To work with the Zarr source code, it is recommended to use `hatch
+<https://hatch.pypa.io/latest/index.html>`_ to create a virtual environment and
+install all Zarr dependencies using the same versions as are used by
 the core developers and continuous integration services. Assuming you have a Python
 3 interpreter already installed, and you have cloned the Zarr source code and your
 current working directory is the root of the repository, you can do something like
 the following::
 
-    $ mkdir -p ~/pyenv/zarr-dev
-    $ python -m venv ~/pyenv/zarr-dev
-    $ source ~/pyenv/zarr-dev/bin/activate
-    $ pip install -e .[test,docs]
+    $ pip install hatch
+    $ hatch env show  # list all available environments
 
-To verify that your development environment is working, you can run the unit tests::
+To verify that your development environment is working, you can run the unit tests
+for one of the test environments, e.g.::
 
-    $ python -m pytest -v tests
+    $ hatch env run --env test.py3.12-2.1-optional run
 
 Creating a branch
 ~~~~~~~~~~~~~~~~~
@@ -143,40 +142,11 @@ spec. The simplest way to run the unit tests is to activate your
 development environment (see `creating a development environment`_ above)
 and invoke::
 
-    $ python -m pytest -v zarr
-
-Some tests require optional dependencies to be installed, otherwise
-the tests will be skipped. To install all optional dependencies, run::
-
-    $ pip install pytest-doctestplus
-
-To also run the doctests within docstrings (requires optional
-dependencies to be installed), run::
-
-    $ python -m pytest -v --doctest-plus zarr
-
-To run the doctests within the tutorial and storage spec (requires
-optional dependencies to be installed), run::
-
-    $ python -m doctest -o NORMALIZE_WHITESPACE -o ELLIPSIS docs/tutorial.rst docs/spec/v2.rst
-
-Note that some tests also require storage services to be running
-locally. To run the Azure Blob Service storage tests, run an Azure
-storage emulator (e.g., azurite) and set the environment variable
-``ZARR_TEST_ABS=1``. If you're using Docker to run azurite, start the service with::
-
-    docker run --rm -p 10000:10000 mcr.microsoft.com/azure-storage/azurite azurite-blob --loose --blobHost 0.0.0.0
-
-To run the Mongo DB storage tests, run a Mongo
-server locally and set the environment variable ``ZARR_TEST_MONGO=1``.
-To run the Redis storage tests, run a Redis server locally on port
-6379 and set the environment variable ``ZARR_TEST_REDIS=1``.
+    $ hatch env run --env test.py3.12-2.1-optional run
 
 All tests are automatically run via GitHub Actions for every pull
 request and must pass before code can be accepted. Test coverage is
-also collected automatically via the Codecov service, and total
-coverage over all builds must be 100% (although individual builds
-may be lower due to Python 2/3 or other differences).
+also collected automatically via the Codecov service.
 
 Code standards - using pre-commit
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -209,11 +179,11 @@ the ``--no-verify`` option with ``git commit``.
 Test coverage
 ~~~~~~~~~~~~~
 
-Zarr maintains 100% test coverage under the latest Python stable release (currently
-Python 3.8). Both unit tests and docstring doctests are included when computing
+Zarr strives to maintain 100% test coverage under the latest Python stable release
+(currently Python 3.12). Both unit tests and docstring doctests are included when computing
 coverage. Running::
 
-    $ python -m pytest -v --cov=zarr --cov-config=pyproject.toml zarr
+    $ hatch env run --env test.py3.12-2.1-optional run-coverage
 
 will automatically run the test suite with coverage and produce a coverage report.
 This should be 100% before code can be accepted into the main code base.
@@ -229,11 +199,7 @@ Docstrings for user-facing classes and functions should follow the
 `numpydoc
 <https://numpydoc.readthedocs.io/en/stable/format.html#docstring-standard>`_
 standard, including sections for Parameters and Examples. All examples
-should run and pass as doctests under Python 3.8. To run doctests,
-activate your development environment, install optional requirements,
-and run::
-
-    $ python -m pytest -v --doctest-plus tests
+should run and pass as doctests under Python 3.10.
 
 Zarr uses Sphinx for documentation, hosted on readthedocs.org. Documentation is
 written in the RestructuredText markup language (.rst files) in the ``docs`` folder.
@@ -245,9 +211,7 @@ notes (``docs/release.rst``).
 
 The documentation can be built locally by running::
 
-    $ cd docs
-    $ make clean; make html
-    $ open _build/html/index.html
+    $ hatch --env docs run build
 
 The resulting built documentation will be available in the ``docs/_build/html`` folder.
 

diff --git a/docs/developers/index.rst b/docs/developers/index.rst
@@ -0,0 +1,10 @@
+
+Developer Guide
+---------------
+
+.. toctree::
+    :maxdepth: 1
+
+    contributing
+    release
+    roadmap
diff --git a/docs/release.rst → docs/developers/release.rst b/docs/release.rst → docs/developers/release.rst
diff --git a/docs/roadmap.rst → docs/developers/roadmap.rst b/docs/roadmap.rst → docs/developers/roadmap.rst
diff --git a/docs/getting_started.rst b/docs/getting_started.rst
diff --git a/docs/guide/index.rst b/docs/guide/index.rst
diff --git a/docs/guide/whatsnew_v3.rst b/docs/guide/whatsnew_v3.rst
diff --git a/docs/index.rst b/docs/index.rst
@@ -8,64 +8,71 @@ Zarr-Python
     :maxdepth: 1
     :hidden:
 
-    getting_started
-    tutorial
-    guide/index
+    quickstart
+    user-guide/index
     api/index
-    release
-    contributing
-    roadmap
+    developers/index
 
 **Version**: |version|
 
 **Download documentation**: `PDF/Zipped HTML <https://readthedocs.org/projects/zarr/downloads/>`_
 
 **Useful links**:
-`Installation <installation.html>`_ |
 `Source Repository <https://github.com/zarr-developers/zarr-python>`_ |
 `Issue Tracker <https://github.com/zarr-developers/zarr-python/issues>`_ |
 `Zulip Chat <https://ossci.zulipchat.com/>`_ |
 `Zarr specifications <https://zarr-specs.readthedocs.io>`_
 
-Zarr is a file storage format for chunked, compressed, N-dimensional arrays based on an open-source specification.
+Zarr is a storage format for chunked, compressed, N-dimensional arrays based on an open-source specification. Highlights include:
+
+* Create N-dimensional arrays with any NumPy dtype.
+* Chunk arrays along any dimension.
+* Compress and/or filter chunks using any Numcodecs_ codec.
+* Store arrays in memory, on disk, inside a Zip file, on S3, ...
+* Read an array concurrently from multiple threads or processes.
+* Write to an array concurrently from multiple threads or processes.
+* Organize arrays into hierarchies via groups.
+
+Zarr-Python Documentation
+-------------------------
 
 .. grid:: 2
 
     .. grid-item-card::
         :img-top: _static/index_getting_started.svg
 
-        Getting Started
-        ^^^^^^^^^^^^^^^
+        Quick Start
+        ^^^^^^^^^^^
 
-        New to Zarr? Check out the getting started guide. It contains an
+        New to Zarr? Check out the quick start guide. It contains a brief
         introduction to Zarr's main concepts and links to additional tutorials.
 
         +++
 
-        .. button-ref:: getting_started
+        .. button-ref:: quickstart
             :expand:
             :color: dark
             :click-parent:
 
-            To the getting started guide
+            To the Quick Start
 
     .. grid-item-card::
         :img-top: _static/index_user_guide.svg
 
-        Tutorial
-        ^^^^^^^^
+        Guide
+        ^^^^^
 
-        The tutorial provides working examples of Zarr classes and functions.
+        The user guide provides a detailed guide for how to use Zarr-Python.
 
         +++
 
-        .. button-ref:: tutorial
+        .. button-ref:: user-guide
             :ref-type: ref
             :expand:
             :color: dark
             :click-parent:
 
-            To the Tutorial
+            To the User Guide
 
     .. grid-item-card::
         :img-top: _static/index_api.svg
@@ -93,13 +100,17 @@ Zarr is a file storage format for chunked, compressed, N-dimensional arrays base
         Contributor's Guide
         ^^^^^^^^^^^^^^^^^^^
 
-        Want to contribute to Zarr? We welcome contributions in the form of bug reports, bug fixes, documentation, enhancement proposals and more. The contributing guidelines will guide you through the process of improving Zarr.
+        Want to contribute to Zarr? We welcome contributions in the form of bug reports,
+        bug fixes, documentation, enhancement proposals and more. The contributing
+        guidelines will guide you through the process of improving Zarr.
 
         +++
 
-        .. button-ref:: contributing
+        .. button-ref:: developers/contributing
             :expand:
             :color: dark
             :click-parent:
 
             To the contributor's guide
+
+.. _Numcodecs: https://numcodecs.readthedocs.io