Skip to content

nan fillvalue attributes written by xarray #176

@veenstrajelmer

Description

@veenstrajelmer

I noticed that xugrid writes nan fillvalue attributes for variables without fillvalue defined. After a quick assessment I see that this is xarray behaviour and it is also happening in their code. A nan fillvalue does not make sense to me and it causes issues in some cases, like this one. In dfm_tools there is a workaround in place that pops nan fillvalue attrs upon reading, but I believe we should avoid writing them. It might be related to xarray not decoding default fillvalues if no fillvalue attribute is present. Either way, let's discuss what is expected/desired behaviour here.

The code below reads a D-FlowFM output which has no nan fillvalue attributes. When writing it with .to_netcdf() there are nan fillvalue attributes. I have tried this also with xugrid testdata, but those datasets already contain nan fillvalue attributes upon reading:

import dfm_tools as dfmt
import xarray as xr
import numpy as np

file_nc = dfmt.data.fm_curvedbend_map(return_filepath=True)
file_out = "temp_fillvals_map.nc"
ds_org = xr.open_dataset(file_nc)
ds_org.to_netcdf(file_out)
ds_out_xr = xr.open_dataset(file_out)

print("nan fillvalue attrs in original dataset:")
ds = ds_org
count_xr = 0
for varn in ds.variables:
    if '_FillValue' in ds[varn].encoding:
        if np.isnan(ds[varn].encoding['_FillValue']):
            print(varn, ds[varn].encoding['_FillValue'])
            count_xr += 1

print()

print("nan fillvalue attrs in dataset written by xugrid/xarray:")
ds = ds_out_xr
count_xr = 0
for varn in ds.variables:
    if '_FillValue' in ds[varn].encoding:
        if np.isnan(ds[varn].encoding['_FillValue']):
            print(varn, ds[varn].encoding['_FillValue'])
            count_xr += 1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions