Closed
Description
What happened:
Created a new conda env for xarray as described in the contribution guide.
Then I installed the h5netcdf and run all the tests (without making any changed in the xarray repo).
Tests failed due to missing dependencies apparently.
What you expected to happen:
Tests should not fail if a package is not a requirement and the backends can operate independently.
Minimal Complete Verifiable Example:
cd xarray
pip install -e .
pip install h5netcdf pytest-xdist
py.test -n 2 .======================================================================== FAILURES =========================================================================
__________________________________________________________ TestH5NetCDFAlreadyOpen.test_deepcopy __________________________________________________________
[gw0] darwin -- Python 3.8.8 /usr/local/anaconda3/envs/xarray/bin/python3.8
self = <xarray.tests.test_backends.TestH5NetCDFAlreadyOpen object at 0x11ce5d5e0>
def test_deepcopy(self):
import h5netcdf
with create_tmp_file() as tmp_file:
> with nc4.Dataset(tmp_file, mode="w") as nc:
E NameError: name 'nc4' is not defined
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_backends.py:2717: NameError
_____________________________________________________ TestH5NetCDFAlreadyOpen.test_open_dataset_group _____________________________________________________
[gw1] darwin -- Python 3.8.8 /usr/local/anaconda3/envs/xarray/bin/python3.8
self = <xarray.tests.test_backends.TestH5NetCDFAlreadyOpen object at 0x11cfe6bb0>
def test_open_dataset_group(self):
import h5netcdf
with create_tmp_file() as tmp_file:
> with nc4.Dataset(tmp_file, mode="w") as nc:
E NameError: name 'nc4' is not defined
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_backends.py:2690: NameError
_____________________________________________________________ test_load_single_value_h5netcdf _____________________________________________________________
[gw0] darwin -- Python 3.8.8 /usr/local/anaconda3/envs/xarray/bin/python3.8
def _get_default_engine_netcdf():
try:
> import netCDF4 # noqa: F401
E ModuleNotFoundError: No module named 'netCDF4'
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:105: ModuleNotFoundError
During handling of the above exception, another exception occurred:
def _get_default_engine_netcdf():
try:
import netCDF4 # noqa: F401
engine = "netcdf4"
except ImportError: # pragma: no cover
try:
> import scipy.io.netcdf # noqa: F401
E ModuleNotFoundError: No module named 'scipy'
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:110: ModuleNotFoundError
During handling of the above exception, another exception occurred:
tmp_path = PosixPath('/private/var/folders/9k/k5388g_s44b_97319sslv9c40000gn/T/pytest-of-Victor/pytest-3/popen-gw0/test_load_single_value_h5netcd0')
@requires_h5netcdf
def test_load_single_value_h5netcdf(tmp_path):
"""Test that numeric single-element vector attributes are handled fine.
At present (h5netcdf v0.8.1), the h5netcdf exposes single-valued numeric variable
attributes as arrays of length 1, as oppesed to scalars for the NetCDF4
backend. This was leading to a ValueError upon loading a single value from
a file, see #4471. Test that loading causes no failure.
"""
ds = xr.Dataset(
{
"test": xr.DataArray(
np.array([0]), dims=("x",), attrs={"scale_factor": 1, "add_offset": 0}
)
}
)
> ds.to_netcdf(tmp_path / "test.nc")
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_backends.py:4993:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/core/dataset.py:1689: in to_netcdf
return to_netcdf(
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:1052: in to_netcdf
engine = _get_default_engine(path_or_file)
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:145: in _get_default_engine
engine = _get_default_engine_netcdf()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def _get_default_engine_netcdf():
try:
import netCDF4 # noqa: F401
engine = "netcdf4"
except ImportError: # pragma: no cover
try:
import scipy.io.netcdf # noqa: F401
engine = "scipy"
except ImportError:
> raise ValueError(
"cannot read or write netCDF files without "
"netCDF4-python or scipy installed"
)
E ValueError: cannot read or write netCDF files without netCDF4-python or scipy installed
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:114: ValueError
==================================================================== warnings summary =====================================================================
xarray/tests/test_accessor_str.py: 18 warnings
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/core/duck_array_ops.py:248: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2))
xarray/tests/test_dataset.py::TestDataset::test_resample_loffset
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_dataset.py:3904: FutureWarning: 'loffset' in .resample() and in Grouper() is deprecated.
>>> df.resample(freq="3s", loffset="8H")
becomes:
>>> from pandas.tseries.frequencies import to_offset
>>> df = df.resample(freq="3s").mean()
>>> df.index = df.index.to_timestamp() + to_offset("8H")
ds.bar.to_series().resample("24H", loffset="-12H").mean()
xarray/tests/test_dataarray.py::TestDataArray::test_resample
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_dataarray.py:2994: FutureWarning: 'loffset' in .resample() and in Grouper() is deprecated.
>>> df.resample(freq="3s", loffset="8H")
becomes:
>>> from pandas.tseries.frequencies import to_offset
>>> df = df.resample(freq="3s").mean()
>>> df.index = df.index.to_timestamp() + to_offset("8H")
expected = DataArray(array.to_series().resample("24H", loffset="-12H").mean())
xarray/tests/test_dataarray.py::TestDataArray::test_polyfit[True-False]
xarray/tests/test_dataarray.py::TestDataArray::test_polyfit[False-False]
/Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/core/dataset.py:6396: RuntimeWarning: overflow encountered in multiply
scale = np.sqrt((lhs * lhs).sum(axis=0))
-- Docs: https://docs.pytest.org/en/stable/warnings.html
================================================================= short test summary info =================================================================
FAILED xarray/tests/test_backends.py::TestH5NetCDFAlreadyOpen::test_deepcopy - NameError: name 'nc4' is not defined
FAILED xarray/tests/test_backends.py::TestH5NetCDFAlreadyOpen::test_open_dataset_group - NameError: name 'nc4' is not defined
FAILED xarray/tests/test_backends.py::test_load_single_value_h5netcdf - ValueError: cannot read or write netCDF files without netCDF4-python or scipy in...
============================== 3 failed, 3474 passed, 4472 skipped, 65 xfailed, 17 xpassed, 22 warnings in 66.86s (0:01:06) ===============================Anything else we need to know?:
Tests still fail when installing scipy only.
They only pass after installing both netcdf4 and scipy.
Environment:
Output of xr.show_versions()
INSTALLED VERSIONS
commit: 48378c4
python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:12:38)
[Clang 11.0.1 ]
python-bits: 64
OS: Darwin
OS-release: 18.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.12.0
libnetcdf: None
xarray: 0.17.1.dev3+g48378c4b
pandas: 1.2.2
numpy: 1.20.1
scipy: None
netCDF4: None
pydap: None
h5netcdf: 0.10.0
h5py: 3.1.0
Nio: None
zarr: None
cftime: 1.4.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.6.0.post20210108
pip: 21.0.1
conda: None
pytest: 6.2.2
IPython: None
sphinx: None