Skip to content

Provide a more informative error when using open_consolidated #776

@andersy005

Description

@andersy005

Problem description

When using open_consolidated on stores that we were created without consolidating the metadata, the user gets a KeyError. I think for folks familiar with zarr, it's easy to diagnose what's going on. However, for new users, the error can be too cryptic (especially when using xarray which exposes zarr.open_consolidated() via a consolidated keyword argument).

I'm wondering if it would be reasonable (if feasible) to provide a more informative error. I am imagining something along these lines:

FileNotFoundError: Couldn't find consolidated metadata file: '.zmetadata'. Try opening the store with `zarr.open(...)`

instead of

KeyError: '.zmetadata'

I don't know if my suggestion is generic enough to warrant a change in zarr. I'm happy to bring up the issue upstream in xarray.

Minimal, reproducible code sample, a copy-pastable example if possible

In [43]: import zarr

In [44]: import numpy as np

In [45]: store = zarr.DirectoryStore("dataset.zarr")

In [46]: ds = zarr.array(np.arange(10))

In [47]: zarr.save(store, ds)

In [48]: zarr.open(store)
Out[48]: <zarr.core.Array (10,) int64>
In [49]: zarr.open_consolidated(store)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-49-f77b4b56ec4c> in <module>
----> 1 zarr.open_consolidated(store)

/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/convenience.py in open_consolidated(store, metadata_key, mode, **kwargs)
   1176 
   1177     # setup metadata store
-> 1178     meta_store = ConsolidatedMetadataStore(store, metadata_key=metadata_key)
   1179 
   1180     # pass through

/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/storage.py in __init__(self, store, metadata_key)
   2767 
   2768         # retrieve consolidated metadata
-> 2769         meta = json_loads(store[metadata_key])
   2770 
   2771         # check format of consolidated metadata

/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/storage.py in __getitem__(self, key)
    844             return self._fromfile(filepath)
    845         else:
--> 846             raise KeyError(key)
    847 
    848     def __setitem__(self, key, value):

KeyError: '.zmetadata'

Version and installation information

Please provide the following:

  • Value of zarr.__version__: 2.8.3
  • Value of numcodecs.__version__: 0.7.3
  • Version of Python interpreter: 3.8.8
  • Operating system: Linux
  • How Zarr was installed: conda

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions