Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for nested dictionaries in atttributes for Zarr #3517

Open
eddienko opened this issue Nov 13, 2019 · 2 comments
Open

Support for nested dictionaries in atttributes for Zarr #3517

eddienko opened this issue Nov 13, 2019 · 2 comments
Labels
contrib-help-wanted topic-zarr Related to zarr storage library

Comments

@eddienko
Copy link

eddienko commented Nov 13, 2019

Similar to #2868, but for Zarr. In this case Zarr should support saving nested dictionaries.

import xarray as xr
ds = xr.Dataset()
ds.attrs['foo'] = {'bar': 1}
ds.to_zarr('foo.zarr', mode='w')

Error message:

TypeError                                 Traceback (most recent call last)
<ipython-input-11-4e7ad4f0a599> in <module>
----> 1 ds.to_zarr('ll.zarr', mode='w')

/srv/conda/envs/notebook/lib/python3.7/site-packages/xarray/core/dataset.py in to_zarr(self, store, mode, synchronizer, group, encoding, compute, consolidated, append_dim)
   1614             compute=compute,
   1615             consolidated=consolidated,
-> 1616             append_dim=append_dim,
   1617         )
   1618 

/srv/conda/envs/notebook/lib/python3.7/site-packages/xarray/backends/api.py in to_zarr(dataset, store, mode, synchronizer, group, encoding, compute, consolidated, append_dim)
   1299     # validate Dataset keys, DataArray names, and attr keys/values
   1300     _validate_dataset_names(dataset)
-> 1301     _validate_attrs(dataset)
   1302 
   1303     if mode == "a":

/srv/conda/envs/notebook/lib/python3.7/site-packages/xarray/backends/api.py in _validate_attrs(dataset)
    209     # Check attrs on the dataset itself
    210     for k, v in dataset.attrs.items():
--> 211         check_attr(k, v)
    212 
    213     # Check attrs on each variable within the dataset

/srv/conda/envs/notebook/lib/python3.7/site-packages/xarray/backends/api.py in check_attr(name, value)
    204                 "a string, an ndarray or a list/tuple of "
    205                 "numbers/strings for serialization to netCDF "
--> 206                 "files".format(value)
    207             )
    208 

TypeError: Invalid value for attr: {'bar': 1} must be a number, a string, an ndarray or a list/tuple of numbers/strings for serialization to netCDF files

Additionally the error message mentions netCDF -- even if I am writing a Zarr file.

Workaround is to save as a list, i.e.

ds.attrs['foo'] = [{'bar': 1}]
@dcherian dcherian added topic-zarr Related to zarr storage library contrib-help-wanted labels Nov 13, 2019
@dcherian
Copy link
Contributor

Thanks for opening an issue @eddienko.

It looks like we'll need to validate attrs based on whether backend="zarr" or backend="netCDF4"

@eddienko
Copy link
Author

Had a go at this -- first PR here, so please review carefully.

eddienko added a commit to eddienko/xarray that referenced this issue Nov 13, 2019
eddienko added a commit to eddienko/xarray that referenced this issue Nov 14, 2019
eddienko added a commit to eddienko/xarray that referenced this issue Nov 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contrib-help-wanted topic-zarr Related to zarr storage library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants