Skip to content

series.to_xarray() fails when MultiIndex not sorted in xarray 0.15.1 #3951

@delgadom

Description

@delgadom

series.to_xarray() fails when MultiIndex not sorted in xarray 0.15.1

Summary

It seems that series.to_xarray() fails (returns incorrect data) in xarray 0.15.1 when the dataframe's MultiIndex dimensions are not sorted

Demonstration

xarray should be able to handle MultiIndices with unsorted dimensions. Using a fresh conda environment with xarray 0.14.1:

$ conda run -n py37xr14 python test.py
>>> df
alpha  B  A
num
0      1  4
1      2  5
2      3  6

>>> df.stack('alpha')
num  alpha
0    B        1
     A        4
1    B        2
     A        5
2    B        3
     A        6
dtype: int64

>>> df.stack('alpha').to_xarray()
<xarray.DataArray (num: 3, alpha: 2)>
array([[1, 4],
       [2, 5],
       [3, 6]])
Coordinates:
  * num      (num) int64 0 1 2
  * alpha    (alpha) object 'B' 'A'

This fails in xarray 0.15.1 - note the data is not merely reordered - the data in column 'B' now has the incorrect values 4, 5, 6 rather than 1, 2, 3:

$ conda run -n py37xr15 python test.py
>>> df
alpha  B  A
num
0      1  4
1      2  5
2      3  6

>>> df.stack('alpha')
num  alpha
0    B        1
     A        4
1    B        2
     A        5
2    B        3
     A        6
dtype: int64

>>> df.stack('alpha').to_xarray()
<xarray.DataArray (num: 3, alpha: 2)>
array([[4, 1],
       [5, 2],
       [6, 3]])
Coordinates:
  * num      (num) int64 0 1 2
  * alpha    (alpha) object 'B' 'A'

Test setup & environment info

contents of test.py
import pandas as pd

df = pd.DataFrame({'B': [1, 2, 3], 'A': [4, 5, 6]})
df = df.rename_axis('num').rename_axis('alpha', axis=1)

print(">>> df")
print(df)

print("\n>>> df.stack('alpha')")
print(df.stack('alpha'))

print("\n>>> df.stack('alpha').to_xarray()")
print(df.stack('alpha').to_xarray())
packages in py37xr14 environment
$ conda list -n py37xr14
# packages in environment at /Users/delgadom/miniconda3/envs/py37xr14:
#
# Name                    Version                   Build  Channel
ca-certificates           2020.4.5.1           hecc5488_0    conda-forge
certifi                   2020.4.5.1       py37hc8dfbb8_0    conda-forge
libblas                   3.8.0               16_openblas    conda-forge
libcblas                  3.8.0               16_openblas    conda-forge
libcxx                    9.0.1                         2    conda-forge
libffi                    3.2.1             h4a8c4bd_1007    conda-forge
libgfortran               4.0.0                         2    conda-forge
liblapack                 3.8.0               16_openblas    conda-forge
libopenblas               0.3.9                h3d69b6c_0    conda-forge
llvm-openmp               9.0.1                h28b9765_2    conda-forge
ncurses                   6.1               h0a44026_1002    conda-forge
numpy                     1.18.1           py37h7687784_1    conda-forge
openssl                   1.1.1f               h0b31af3_0    conda-forge
pandas                    1.0.3            py37h94625e5_0    conda-forge
pip                       20.0.2                     py_2    conda-forge
python                    3.7.6           h90870a6_5_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.7                     1_cp37m    conda-forge
pytz                      2019.3                     py_0    conda-forge
readline                  8.0                  hcfe32e1_0    conda-forge
setuptools                46.1.3           py37hc8dfbb8_0    conda-forge
six                       1.14.0                     py_1    conda-forge
sqlite                    3.30.1               h93121df_0    conda-forge
tk                        8.6.10               hbbe82c9_0    conda-forge
wheel                     0.34.2                     py_1    conda-forge
xarray                    0.14.1                     py_1    conda-forge
xz                        5.2.5                h0b31af3_0    conda-forge
zlib                      1.2.11            h0b31af3_1006    conda-forge
packages in py37xr15 environment
$ conda list -n py37xr15
# packages in environment at /Users/delgadom/miniconda3/envs/py37xr15:
#
# Name                    Version                   Build  Channel
ca-certificates           2020.4.5.1           hecc5488_0    conda-forge
certifi                   2020.4.5.1       py37hc8dfbb8_0    conda-forge
libblas                   3.8.0               16_openblas    conda-forge
libcblas                  3.8.0               16_openblas    conda-forge
libcxx                    9.0.1                         2    conda-forge
libffi                    3.2.1             h4a8c4bd_1007    conda-forge
libgfortran               4.0.0                         2    conda-forge
liblapack                 3.8.0               16_openblas    conda-forge
libopenblas               0.3.9                h3d69b6c_0    conda-forge
llvm-openmp               9.0.1                h28b9765_2    conda-forge
ncurses                   6.1               h0a44026_1002    conda-forge
numpy                     1.18.1           py37h7687784_1    conda-forge
openssl                   1.1.1f               h0b31af3_0    conda-forge
pandas                    1.0.3            py37h94625e5_0    conda-forge
pip                       20.0.2                     py_2    conda-forge
python                    3.7.6           h90870a6_5_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.7                     1_cp37m    conda-forge
pytz                      2019.3                     py_0    conda-forge
readline                  8.0                  hcfe32e1_0    conda-forge
setuptools                46.1.3           py37hc8dfbb8_0    conda-forge
six                       1.14.0                     py_1    conda-forge
sqlite                    3.30.1               h93121df_0    conda-forge
tk                        8.6.10               hbbe82c9_0    conda-forge
wheel                     0.34.2                     py_1    conda-forge
xarray                    0.15.1                     py_0    conda-forge
xz                        5.2.5                h0b31af3_0    conda-forge
zlib                      1.2.11            h0b31af3_1006    conda-forge

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions