Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sortby descending does not handle nans correctly #7358

Open
4 tasks done
blackbox-tech opened this issue Dec 5, 2022 · 1 comment
Open
4 tasks done

sortby descending does not handle nans correctly #7358

blackbox-tech opened this issue Dec 5, 2022 · 1 comment

Comments

@blackbox-tech
Copy link

What happened?

sortby ascending=False just returns the reversed output of ascending=True
This is incorrect if the array contains nan values. (< is numerically not the opposite of > for nan values)

For example:

a = xr.DataArray([3, np.nan, 4, 2, np.nan], 
                 {"label": ["a", "b", "c", "d", "e"]})
a.sortby(a, ascending=False)

produces

[nan, nan,  4.,  3.,  2.]

What did you expect to happen?

Using the example above, I expect to see:

[ 4.,  3.,  2., nan, nan]

A workaround to return the correct result can be implemented like this:

idx = np.argpartition(-a, np.arange((np.isfinite(a)).sum())).values
a[idx]

Minimal Complete Verifiable Example

No response

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

xarray                    2022.11.0       py310hecd8cb5_0
@blackbox-tech blackbox-tech added bug needs triage Issue that has not been reviewed by xarray team member labels Dec 5, 2022
@dcherian
Copy link
Contributor

Thanks @blackbox-tech . We'll need to fix it here but using lexsort to preserve sorting by multiple arrays:

xarray/xarray/core/dataset.py

Lines 6992 to 6996 in db68db6

indices = {}
for key, arrays in vars_by_dim.items():
order = np.lexsort(tuple(reversed(arrays)))
indices[key] = order if ascending else order[::-1]
return aligned_self.isel(indices)

@dcherian dcherian added contrib-help-wanted and removed needs triage Issue that has not been reviewed by xarray team member labels Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants