trouble loading netcdf4 files with xarray on s3

[scottyhq]

I'm working on allowing direct access to netcdf4/hdf5 file-like objects (pydata/xarray#2782). This seems to be working fine with gcsfs, but not s3fs (versions 0.2 from conda-forge). Here is a gist with the relevant code and error traceback:

https://gist.github.com/scottyhq/304a3c4b4e198776b8d82fb3a9f300e3

and an abbreviated traceback here:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/Documents/GitHub/xarray/xarray/backends/file_manager.py in acquire(self, needs_lock)
    166             try:
--> 167                 file = self._cache[self._key]
    168             except KeyError:

~/Documents/GitHub/xarray/xarray/backends/lru_cache.py in __getitem__(self, key)
     40         with self._lock:
---> 41             value = self._cache[key]
     42             self._cache.move_to_end(key)

KeyError: [<function _open_h5netcdf_group at 0x11d8b0ae8>, (<S3File grfn-content-prod/S1-GUNW-A-R-137-tops-20181129_20181123-020010-43220N_41518N-PP-e2c7-v2_0_0.nc>,), 'r', (('group', '/science/grids/data'),)]

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/test_env/lib/python3.6/site-packages/s3fs/core.py in readinto(self, b)
   1498         data = self.read()
-> 1499         b[:len(data)] = data
   1500         return len(data)

~/miniconda3/envs/test_env/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-darwin.so in View.MemoryView.memoryview.__setitem__()

~/miniconda3/envs/test_env/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-darwin.so in View.MemoryView.memoryview.setitem_slice_assignment()

~/miniconda3/envs/test_env/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-darwin.so in View.MemoryView.memoryview_copy_contents()

~/miniconda3/envs/test_env/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-darwin.so in View.MemoryView._err_extents()

ValueError: got differing extents in dimension 0 (got 8 and 59941567)

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/test_env/lib/python3.6/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

any guidance as to what might be going on here would be appreciated!

[martindurant]

That is quite a traceback!
I am surprised that gcsfs worked, rather than that s3fs did not - hdf5 is a C-level reader that likes to have a local real file to read from. I have heard that they have tried to allow for python file-like objects, but I don't know how that's implemented - apparently something is subtly different between the two file implementation classes.

[leroygr]

Same problem for me, I can't read a netCDF on S3 using s3fs with h5netcdf:

>>> s3 = s3fs.S3FileSystem(key=os.environ['AWS_DS_AGENT_KEY_ID'],
                    secret=os.environ['AWS_DS_AGENT_ACCESS_KEY'])
>>> fileobj = s3.open(s3_fp)
>>> nc = h5netcdf.File(fileobj,'r', invalid_netcdf=True)

Traceback

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in readinto(self, b)
   1498         data = self.read()
-> 1499         b[:len(data)] = data
   1500         return len(data)

~/miniconda3/envs/s3test/lib/python3.7/site-packages/h5py/h5fd.cpython-37m-x86_64-linux-gnu.so in View.MemoryView.memoryview.__setitem__()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/h5py/h5fd.cpython-37m-x86_64-linux-gnu.so in View.MemoryView.memoryview.setitem_slice_assign_scalar()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/h5py/h5fd.cpython-37m-x86_64-linux-gnu.so in View.MemoryView._memoryviewslice.assign_item_from_object()

TypeError: an integer is required

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')

SystemError: PyEval_EvalFrameEx returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
<ipython-input-7-49848b8cfbdc> in <module>
----> 1 nc = h5netcdf.File(fileobj,'r', invalid_netcdf=True)

~/miniconda3/envs/s3test/lib/python3.7/site-packages/h5netcdf/core.py in __init__(self, path, mode, invalid_netcdf, **kwargs)
    603                 else:
    604                     self._preexisting_file = mode in {'r', 'r+', 'a'}
--> 605                     self._h5file = h5py.File(path, mode, **kwargs)
    606         except Exception:
    607             self._closed = True

~/miniconda3/envs/s3test/lib/python3.7/site-packages/h5py/_hl/files.py in __init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, **kwds)
    392                 fid = make_fid(name, mode, userblock_size,
    393                                fapl, fcpl=make_fcpl(track_order=track_order),
--> 394                                swmr=swmr)
    395 
    396             if swmr_support:

~/miniconda3/envs/s3test/lib/python3.7/site-packages/h5py/_hl/files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    168         if swmr and swmr_support:
    169             flags |= h5f.ACC_SWMR_READ
--> 170         fid = h5f.open(name, flags, fapl=fapl)
    171     elif mode == 'r+':
    172         fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5f.pyx in h5py.h5f.open()

h5py/defs.pyx in h5py.defs.H5Fopen()

h5py/h5fd.pyx in h5py.h5fd.H5FD_fileobj_read()

~/miniconda3/envs/s3test/lib/python3.7/site-packages/s3fs/core.py in seek(self, loc, whence)
   1234             from start of file, current location or end of file, resp.
   1235         """
-> 1236         if not self.readable():
   1237             raise ValueError('Seek only available in read mode')
   1238         if whence == 0:

[martindurant]

If you can post the file somewhere public, I can try to find out what's going on.

[pbranson]

I am keen to see a way to do this without a fuse mount - here is an open file:

import xarray as xr
import s3fs
fs = s3fs.S3FileSystem(anon=True)
s3path = 'imos-data/IMOS/SRS/OC/gridded/aqua/P1D/2010/05/A.P1D.20100507T000000Z.aust.ipar.nc'

fobj = fs.open(s3path)
ds = xr.open_dataset(fobj,engine='h5netcdf')

Produces roughly the same stack trace

[martindurant]

I am getting

      5
      6 fobj = fs.open(s3path)
----> 7 ds = xr.open_dataset(fobj,engine='h5netcdf')

~/anaconda/envs/py36/lib/python3.6/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, autoclose, concat_characters, decode_coords, engine, chunks, lock, cache, drop_variables, backend_kwargs)
    345     else:
    346         if engine is not None and engine != 'scipy':
--> 347             raise ValueError('can only read file-like objects with '
    348                              "default engine or engine='scipy'")
    349         # assume filename_or_obj is a file-like object

ValueError: can only read file-like objects with default engine or engine='scipy'

however, the problem for you seems to be here:

-> 1499         b[:len(data)] = data

The message implies that that the data being inserted is the wrong size; it would be good to debug at that point to see what the buffer b and data contain.

[pbranson]

Yes - I think there were some recent updates that changed this behaviour to allow h5netcdf with file-like objects I was using these versions from pip to get the previously mentioned error: h5netcdf-0.7.1 h5py-2.9.0 pytz-2018.9 xarray-0.12.0

…

On Wed, Apr 3, 2019 at 9:36 AM Martin Durant ***@***.***> wrote: I am getting 5 6 fobj = fs.open(s3path) ----> 7 ds = xr.open_dataset(fobj,engine='h5netcdf') ~/anaconda/envs/py36/lib/python3.6/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, autoclose, concat_characters, decode_coords, engine, chunks, lock, cache, drop_variables, backend_kwargs) 345 else: 346 if engine is not None and engine != 'scipy': --> 347 raise ValueError('can only read file-like objects with ' 348 "default engine or engine='scipy'") 349 # assume filename_or_obj is a file-like object ValueError: can only read file-like objects with default engine or engine='scipy' however, the problem for you seems to be here: -> 1499 b[:len(data)] = data The message implies that that the data being inserted is the wrong size; it would be good to debug at that point to see what the buffer b and data contain. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#168 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AM3bQKm461Vg3QIEDlOcEDeoWk8xFVfOks5vdAWwgaJpZM4bTe4Y> .

[pbranson]

You are correct - if I check back through the stack trace I am getting the error here:

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in readinto(self, b)
   1498         data = self.read()
-> 1499         b[:len(data)] = data
   1500         return len(data)

/opt/conda/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-x86_64-linux-gnu.so in View.MemoryView.memoryview.__setitem__()

/opt/conda/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-x86_64-linux-gnu.so in View.MemoryView.memoryview.setitem_slice_assignment()

/opt/conda/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-x86_64-linux-gnu.so in View.MemoryView.memoryview_copy_contents()

/opt/conda/lib/python3.6/site-packages/h5py/h5fd.cpython-36m-x86_64-linux-gnu.so in View.MemoryView._err_extents()

ValueError: got differing extents in dimension 0 (got 8 and 24247921)

I will have a go at setting a breakpoint there and taking a look

[pbranson]

Not entirely clear to me how all these libraries tie together but it seems that

h5py:files.py 170 fid = h5f.open(name, flags, fapl=fapl)

is calling out to the hdf5 c-library function "H5Fopen", which is expecting a string filepath, where as name at this point is an s3fs.S3File object. Somehow passing the "name" parameter is invoking

s3fs:core.py

1497        def readinto(self, b):
                    data = self.read()
                    b[:len(data)] = data
                    return len(data)

where b is a <MemoryView of 'array' at 0x1b458a73be0> but I cant workout where b is instantiated - it only has a length of 8, where as the binary data read from s3 is much larger - hence the exception.

However even if that succeeded I dont know how this would work anyway, given the c library is expecting a string file path, rather than a binary memory view?

I am surprised that gcsfs worked, rather than that s3fs did not - hdf5 is a C-level reader that likes to have a local real file to read from.

If I get time I will take a look at why the gcsfs is working

[martindurant]

OK, so something "new" :)
I would suspect that the memoryview has a complex type other than bytes, and s3fs is trying to fill the buffer with bytes (although it doesn't appear to be an exact multiple). readinto is very rarely used anywhere, surprised to see it, but I suppose the memory must have been allocated in C-land.

btw: the difficulties with hdf are the main reason for interest in libraries like zarr (or zarr as a backend for netcdf), which is known to work well with s3fs/gcsfs/etc. It may or may not be a viable alternative for you.

[pbranson]

Thanks Martin - testing out a 'simple' (albeit slow) way converting of converting netCDF that is already present in cloud storage into Zarr format, with out formally 'mirroring' the files locally somewhere. Noting that if performance is crucial and you have the AWS budget, then setting up AWS FSx is the obvious way to go (i.e. https://jiaweizhuang.github.io/blog/fsx-experiments/)

…

On Wed, Apr 3, 2019 at 10:32 PM Martin Durant ***@***.***> wrote: OK, so something "new" :) I would suspect that the memoryview has a complex type other than bytes, and s3fs is trying to fill the buffer with bytes (although it doesn't appear to be an exact multiple). readinto is very rarely used anywhere, surprised to see it, but I suppose the memory must have been allocated in C-land. btw: the difficulties with hdf are the main reason for interest in libraries like zarr (or zarr as a backend for netcdf), which is known to work well with s3fs/gcsfs/etc. It may or may not be a viable alternative for you. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#168 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AM3bQDEM5VDX0QSbBQ8-XDMwLV1dDS0yks5vdLttgaJpZM4bTe4Y> .

[rsignell-usgs]

@martindurant, I'm also hitting this as well. Was hoping to follow up on @rabernat's suggestion to include this in our testing of different options for accessing NetCDF4/HDF5 on s3 (in addition to Zarr and HSDS).

I got the same error you did:

ValueError: can only read file-like objects with default engine or engine='scipy'

when I forgot to install h5netcdf and h5py into my environment.

[martindurant]

@pbranson , did you manage to learn anything about this issue?

[martindurant]

Hm, with everything updated (h5netcdf , h5py, s3fs), the first invocation just worked:

In [5]: fs = s3fs.S3FileSystem(anon=True)
   ...: s3path = 'imosdata/IMOS/SRS/OC/gridded/aqua/P1D/2010/05/A.P1D.20100507T000000Z.aust.ipar.nc'
   ...:
   ...: fobj = fs.open(s3path)
In [6]: nc = h5netcdf.File(fobj, 'r', invalid_netcdf=True)
In [7]: nc
Out[7]:
<h5netcdf.File 'A.P1D.20100507T000000Z.aust.ipar.nc>' (mode r)>
Dimensions:
    latitude: 7001
    longitude: 10001
    time: 1
Groups:
Variables:
    time: ('time',) float64
    latitude: ('latitude',) float64
    longitude: ('longitude',) float64
    ipar: ('time', 'latitude', 'longitude') float32
Attributes:
    history: b'File initialised at 2015-12-17T19:03:50.793738\nInitialised var ipar at 2015-12-17T19:04:36.563452\nAdd Granule A20100507_0230.20150923161152.L2OC_BASE.ipar.nc at 2015-12-17T19:04:38.498914\nAdd Granule A20100507_0235.20150923151200.L2OC_BASE.ipar.nc at 2015-12-17T19:04:38.975299\nAdd Granule A20100507_0240.20150923134822.L2OC_BASE.ipar.nc at 2015-12-17T19:04:39.483551\nAdd Granule A20100507_0245.20150923143121.L2OC_BASE.ipar.nc at 2015-12-17T19:04:39.793043\nAdd Granule A20100507_0405.20150923141146.L2OC_BASE.ipar.nc at 2015-12-17T19:04:40.401902\nAdd Granule A20100507_0410.20150923162326.L2OC_BASE.ipar.nc at 2015-12-17T19:04:40.977119\nAdd Granule A20100507_0415.20150923133857.L2OC_BASE.ipar.nc at 2015-12-17T19:04:41.430398\nAdd Granule A20100507_0420.20150923150036.L2OC_BASE.ipar.nc at 2015-12-17T19:04:41.923474\nAdd Granule A20100507_0540.20150923152402.L2OC_BASE.ipar.nc at 2015-12-17T19:04:42.336277\nAdd Granule A20100507_0545.20150923154421.L2OC_BASE.ipar.nc at 2015-12-17T19:04:43.116328\nAdd Granule A20100507_0550.20150923140042.L2OC_BASE.ipar.nc at 2015-12-17T19:04:43.709527\nAdd Granule A20100507_0555.20150923155628.L2OC_BASE.ipar.nc at 2015-12-17T19:04:44.321537\nAdd Granule A20100507_0600.20150923165701.L2OC_BASE.ipar.nc at 2015-12-17T19:04:44.871419\nAdd Granule A20100507_0720.20150923142308.L2OC_BASE.ipar.nc at 2015-12-17T19:04:45.394833\nAdd Granule A20100507_0725.20150923132636.L2OC_BASE.ipar.nc at 2015-12-17T19:04:46.131246\nAdd Granule A20100507_0730.20150923163350.L2OC_BASE.ipar.nc at 2015-12-17T19:04:46.614609\nAdd Granule A20100507_0735.20150923153102.L2OC_BASE.ipar.nc at 2015-12-17T19:04:47.083167\nAdd Granule A20100507_0740.20150923144622.L2OC_BASE.ipar.nc at 2015-12-17T19:04:47.608014'
    Conventions: b'CF-1.6'
    source_path: b'imos-srs/archive/oc/aqua/1d/v201508/2010/05/A20100507.L2OC_BASE.aust.ipar.ncimos-srs/archive/oc/aqua/1d/v201508/2010/05/A20100507.L2OC_BASE.aust.ipar.ncimos-srs/archive/oc/aqua/1d/v201508/2010/05/A20100507.L2OC_BASE.aust.ipar.nc'

but via xarray it does not.