You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, when objects with with non-unique (duplicated) values in one of their indexes are passed to align, an error surfaces from pandas: InvalidIndexError: Reindexing only valid with uniquely valued Index objects
We could certainly give a more informative error here (see this complaint on StackOverflow), but a bigger issue is that this probably isn't strictly necessary. Instead, we could skip indexes for alignment if they are already equal. This is slightly less principled (a non-unique index may indicate something has gone wrong), but certainly more convenient and more inline with how pandas works (e.g., it even allows arithmetic between objects with non-unique indexes, which I believe does not work currently in xarray).
Currently, we do this as a special case when merging arrays and exactly one has labels (see _align_for_merge in #950). But we could probably do this in general, either by default or with a flag to enable it (or turn it off). This would then propagate to every xarray operation that uses align under the covers.
The text was updated successfully, but these errors were encountered:
I think it's important to preserve the ability to work with non-unique indexes insofar as it is necessary for cleaning your data. That's why we support non-unique indexes in merge, which is internally used anytime you assign to a Dataset. I'm less sure about the use cases for arithmetic and the like.
I just checked on xarray v0.7.2 and the example from the StackOverflow post worked. I think we should treat this as a regression -- we were already doing this prior to v0.8.
Currently, when objects with with non-unique (duplicated) values in one of their indexes are passed to align, an error surfaces from pandas:
InvalidIndexError: Reindexing only valid with uniquely valued Index objects
We could certainly give a more informative error here (see this complaint on StackOverflow), but a bigger issue is that this probably isn't strictly necessary. Instead, we could skip indexes for alignment if they are already equal. This is slightly less principled (a non-unique index may indicate something has gone wrong), but certainly more convenient and more inline with how pandas works (e.g., it even allows arithmetic between objects with non-unique indexes, which I believe does not work currently in xarray).
Currently, we do this as a special case when merging arrays and exactly one has labels (see
_align_for_merge
in #950). But we could probably do this in general, either by default or with a flag to enable it (or turn it off). This would then propagate to every xarray operation that uses align under the covers.The text was updated successfully, but these errors were encountered: