Multiple unstack in multi level columns DataFrame

[faulaire]

Code Sample, a copy-pastable example if possible

import pandas as pd
from six import StringIO
stats = pd.DataFrame({('B', 'C'): 
                          {
                              (10, 20, 30): 0.0,
                              (10, 20, 40): 0.0
                          },
                      ('B', 'D'): 
                          {
                              (10, 20, 30): 0.0, 
                              (10, 20, 40): 0.0
                          }
                     })

stats.index.names = ['i1', 'i2', 'i3']
stats.columns.names = ['c1', 'c2']

# Expected true,  actual false
stats.unstack(['i2', 'i1']).columns.names[-2:] == ['i2', 'i1']

# In case of single level column, expected true, actual true
stats['B'].unstack(['i2', 'i1']).columns.names[-2:] == ['i2', 'i1']

Problem description

Trying to unstack multiindex multicolumns DataFrame, yield to an unexpected behavior.
The unstacked levels are not those expected (see the code above).

Note: In a single level column case, behavior is the expected one.

Expected Output

N/A

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.13.final.0
python-bits: 64
OS: Linux
OS-release: 2.6.32-642.6.2.el6.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.23.2
pytest: 3.2.3
pip: 9.0.1
setuptools: 36.3.0
Cython: 0.27.3
numpy: 1.13.3
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 5.1.0
sphinx: 1.4.4
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.2.1
numexpr: 2.6.5
feather: None
matplotlib: 2.1.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.9999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[gfyoung]

cc @jreback @toobaz

[gfyoung]

@faulaire : Do you mind modifying your example slightly to construct the DataFrame explicitly instead of via read_csv ? That way, it makes it clear to everyone that we're just focusing on the (un)stacking, and that CSV functionality is not really involved in this issue.

[faulaire]

@gfyoung : done.

[alexmemory]

Here's a different example of unexpected unstack behavior that may be related. Like @faulaire's example, for single level columns the behavior is the expected one.

import pandas as pd
df = (pd.DataFrame({**{'a'+str(n):[1] for n in range(7)}, 
                    **{'b0':[2]}})
      .set_index(['a'+str(n) for n in range(7)]))
df = pd.concat([df], keys=['b1'], axis=1)
# Expected true, actual true
df.unstack(['a0', 'a1', 'a2']).index.names == ['a3', 'a4', 'a5', 'a6']
# Expected true, actual false
df.unstack(['a0', 'a1', 'a2', 'a3']).index.names == ['a4', 'a5', 'a6']
# Expected false, actual true
df.unstack(['a0', 'a1', 'a2', 'a3']).index.names == ['a3', 'a5', 'a6']
# Expected true, actual false
df.unstack(['a0', 'a1', 'a2', 'a3', 'a4']).index.names == ['a5', 'a6']
# Expected false, actual true
df.unstack(['a0', 'a1', 'a2', 'a3', 'a4']).index.names == ['a3', 'a6']
# If single level column, no problem 
df.columns = df.columns.droplevel()
# Expected true, actual true
df.unstack(['a0', 'a1', 'a2', 'a3']).index.names == ['a4', 'a5', 'a6']