Closed
Description
When packaging zarr 2.9.3 for running the test suite it fails in test_storage.py::TestFSStore::test_create:
[ 27s] + PYTHONPATH=/home/abuild/rpmbuild/BUILDROOT/python-zarr-2.9.3-0.x86_64/usr/lib/python3.9/site-packages
[ 27s] + PYTHONDONTWRITEBYTECODE=1
[ 27s] + pytest-3.9 --ignore=_build. --ignore=_build.python39 --ignore=_build.python38 -v
[ 27s] ============================= test session starts ==============================
[ 27s] platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.9
[ 27s] cachedir: .pytest_cache
[ 27s] rootdir: /home/abuild/rpmbuild/BUILD/zarr-2.9.3, configfile: pytest.ini
[ 32s] collecting ... collected 2447 items
[ 32s]
[ ... snip a zillion passing tests ...]
[ 156s]
[ 146s]
[ 146s] =================================== FAILURES ===================================
[ 146s] ___________________________ TestFSStore.test_create ____________________________
[ 146s]
[ 146s] self = <zarr.tests.test_storage.TestFSStore object at 0x7f1473e8ba00>
[ 146s]
[ 146s] def test_create(self):
[ 146s] import zarr
[ 146s] path1 = tempfile.mkdtemp()
[ 146s] path2 = tempfile.mkdtemp()
[ 146s] g = zarr.open_group("file://" + path1, mode='w',
[ 146s] storage_options={"auto_mkdir": True})
[ 146s] a = g.create_dataset("data", shape=(8,))
[ 146s] a[:4] = [0, 1, 2, 3]
[ 146s] assert "data" in os.listdir(path1)
[ 146s] assert ".zgroup" in os.listdir(path1)
[ 146s]
[ 146s] g = zarr.open_group("simplecache::file://" + path1, mode='r',
[ 146s] storage_options={"cache_storage": path2,
[ 146s] "same_names": True})
[ 146s] > assert g.data[:].tolist() == [0, 1, 2, 3, 0, 0, 0, 0]
[ 146s]
[ 146s] zarr/tests/test_storage.py:975:
[ 146s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 146s] zarr/core.py:662: in __getitem__
[ 146s] return self.get_basic_selection(selection, fields=fields)
[ 146s] zarr/core.py:787: in get_basic_selection
[ 146s] return self._get_basic_selection_nd(selection=selection, out=out,
[ 146s] zarr/core.py:830: in _get_basic_selection_nd
[ 146s] return self._get_selection(indexer=indexer, out=out, fields=fields)
[ 146s] zarr/core.py:1125: in _get_selection
[ 146s] self._chunk_getitems(lchunk_coords, lchunk_selection, out, lout_selection,
[ 146s] zarr/core.py:1836: in _chunk_getitems
[ 146s] cdatas = self.chunk_store.getitems(ckeys, on_error="omit")
[ 146s] zarr/storage.py:1111: in getitems
[ 146s] results = self.map.getitems(keys_transformed, on_error="omit")
[ 146s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 146s]
[ 146s] self = <fsspec.mapping.FSMap object at 0x7f1474185250>, keys = ['data/0']
[ 146s] on_error = 'omit'
[ 146s]
[ 146s] def getitems(self, keys, on_error="raise"):
[ 146s] """Fetch multiple items from the store
[ 146s]
[ 146s] If the backend is async-able, this might proceed concurrently
[ 146s]
[ 146s] Parameters
[ 146s] ----------
[ 146s] keys: list(str)
[ 146s] They keys to be fetched
[ 146s] on_error : "raise", "omit", "return"
[ 146s] If raise, an underlying exception will be raised (converted to KeyError
[ 146s] if the type is in self.missing_exceptions); if omit, keys with exception
[ 146s] will simply not be included in the output; if "return", all keys are
[ 146s] included in the output, but the value will be bytes or an exception
[ 146s] instance.
[ 146s]
[ 146s] Returns
[ 146s] -------
[ 146s] dict(key, bytes|exception)
[ 146s] """
[ 146s] keys2 = [self._key_to_str(k) for k in keys]
[ 146s] oe = on_error if on_error == "raise" else "return"
[ 146s] try:
[ 146s] out = self.fs.cat(keys2, on_error=oe)
[ 146s] except self.missing_exceptions as e:
[ 146s] raise KeyError from e
[ 146s] out = {
[ 146s] k: (KeyError() if isinstance(v, self.missing_exceptions) else v)
[ 146s] > for k, v in out.items()
[ 146s] }
[ 146s] E AttributeError: 'bytes' object has no attribute 'items'
[ 146s]
[ 146s] /usr/lib/python3.9/site-packages/fsspec/mapping.py:95: AttributeError
[ 146s] =============================== warnings summary ===============================
[ 146s] zarr/tests/test_creation.py::test_open_array
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite_group
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_init_array
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_init_array_path
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite_path
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite_chunk_store
[ 146s] zarr/tests/test_storage.py::TestN5Store::test_filters
[ 146s] /home/abuild/rpmbuild/BUILD/zarr-2.9.3/zarr/n5.py:420: RuntimeWarning: Not all N5 implementations support blosc compression (yet). You might not be able to open the dataset with another N5 library.
[ 146s] warnings.warn(
[ 146s]
[ 146s] zarr/tests/test_creation.py::test_compression_args
[ 146s] /home/abuild/rpmbuild/BUILD/zarr-2.9.3/zarr/tests/test_creation.py:472: UserWarning: 'compression' keyword argument overridden by 'compressor'
[ 146s] z = create(100, compressor=Zlib(9), compression='bz2', compression_opts=1)
[ 146s]
[ 146s] zarr/tests/test_creation.py::test_compression_args
[ 146s] /home/abuild/rpmbuild/BUILD/zarr-2.9.3/zarr/tests/test_creation.py:472: UserWarning: 'compression_opts' keyword argument overridden by 'compressor'
[ 146s] z = create(100, compressor=Zlib(9), compression='bz2', compression_opts=1)
[ 146s]
[ 146s] zarr/tests/test_creation.py::test_compression_args
[ 146s] /home/abuild/rpmbuild/BUILD/zarr-2.9.3/zarr/tests/test_creation.py:478: UserWarning: 'compression_opts' keyword argument overridden by 'compressor'
[ 146s] z = create(100, compressor=Zlib(9), compression_opts=1)
[ 146s]
[ 146s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[ 146s] ============================= slowest 10 durations =============================
[ 146s] 7.74s call zarr/tests/test_indexing.py::test_set_orthogonal_selection_3d
[ 146s] 7.51s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_resize_2d
[ 146s] 7.48s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_iter
[ 146s] 6.77s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_islice
[ 146s] 4.69s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_array_order
[ 146s] 4.35s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_append_2d_axis
[ 146s] 4.04s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_append_2d
[ 146s] 2.69s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_np_ufuncs
[ 146s] 2.28s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_dtypes
[ 146s] 1.95s call zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_compressors
[ 146s] =========================== short test summary info ============================
[ 146s] FAILED zarr/tests/test_storage.py::TestFSStore::test_create - AttributeError:...
[ 146s] = 1 failed, 2031 passed, 413 skipped, 2 xfailed, 11 warnings in 118.60s (0:01:58) =
If could be Python 3.9 issue (hello, #629 !), but I think it should be unstable API of whatever package provides self.fs.cat.
Version and installation information
Please provide the following:
- Value of
zarr.__version__2.9.3 - Value of
numcodecs.__version__0.7.3 - Version of Python interpreter 3.9.6
- Operating system (Linux/Windows/Mac) Linux, packaging for openSUSE/Tumbleweed
- How Zarr was installed (e.g., "using pip into virtual environment", or "using conda") tarball
Complete build log with all versions of packages used and steps taken.
Metadata
Metadata
Assignees
Labels
No labels