rasterio.open; attrs remain empty

[Manangka]

In GitLab by @tomvansteijn on Feb 11, 2021, 09:45

When opening an integer raster file (gtiff), the crs and nodata value are properly set in the DataArray attrs dict. However, when opening another raster file, a float raster, the attrs remain empty.

Not sure if the dtype makes the difference. When the attrs are not set, the "dmin" and "dmax" properties are not calculated correctly as the raster nodata value is not taken into account. This shows in the colormap in the iMOD viewer.

Attached a raster file for which the attrs remain empty when calling rasterio.open.

recharge_mmd.zip

[Manangka]

In GitLab by @Huite on Mar 8, 2021, 10:38

Thanks for the heads-up and the example; I'll have a look at it this week.
I think we automatically convert to NaN for floating point data (since that's basically the xarray default), but it should still set the nodata value in the attrs. It's been a while since I last looked at this code.

[Manangka]

In GitLab by @JoerivanEngelen on May 12, 2021, 15:50

I investigated this, and indeed it is caused by the datatype.
What happens is that we do nodata = attrs.pop("nodata") if float in _dask

...
    if attrs is None:
        attrs = header(path, pattern)
    # If we don't unpack, it seems we run into trouble with the dask array later
    # on, probably because attrs isn't immutable. This works fine instead.
    headersize = attrs.pop("headersize")
    nrow = attrs["nrow"]
    ncol = attrs["ncol"]
    dtype = attrs["dtype"]
    # In case of floating point data, nodata is always represented by nan.
    if "float" in dtype:
        nodata = attrs.pop("nodata")
    else:
        nodata = attrs["nodata"]
...

Not popping this is probably a bad idea, I reckon, because it can lead to confusion as in the data nodata is in fact a np.nan. We could add np.nan to the attrs["nodata"] as a fix. @Huite , what do you think?

When you write to .IDF, nodata is by default set to 1.0e20. You can set this to something to your liking with:
imod.idf.save(path, a, nodata=-9999.). This is stored in the .IDF, so I'd expect the iMOD viewer can handle it.