REGR: CategoricalIndex and IntervalIndex are missing _index_data attribute

[jschendel]

It looks like the _index_data attribute wasn't added to CategoricalIndex or IntervalIndex. This leads to some regressions with GroupBy.apply.

Not sure what the fix should be, as there isn't a singular array that directly backs the underlying values for CategoricalIndex or IntervalIndex, so the approach used to add _index_data for other types doesn't look like it will work here.

Example of broken behavior on master:

In [1]: import pandas as pd; pd.__version__
Out[1]: '1.0.0rc0+146.g6f395ad42'

In [2]: index = pd.CategoricalIndex(list("abc"))  
   ...: df = pd.DataFrame({"group": [0, 0, 1], "value": [1, 2, 3]}, index=index)

In [3]: df.groupby("group").apply(lambda x: x)
---------------------------------------------------------------------------
AttributeError: 'CategoricalIndex' object has no attribute '_index_data'

Same behavior worked on 0.25.3:

In [1]: import pandas as pd; pd.__version__
Out[1]: '0.25.3'

In [2]: index = pd.CategoricalIndex(list("abc"))  
   ...: df = pd.DataFrame({"group": [0, 0, 1], "value": [1, 2, 3]}, index=index)

In [3]: df.groupby("group").apply(lambda x: x)
Out[3]: 
   group  value
a      0      1
b      0      2
c      1      3

Swapping CategoricalIndex with IntervalIndex results in the same behavior above.

[TomAugspurger]

@jbrockmendel do you have thoughts on how to fix this?

[jschendel]

@TomAugspurger : This can be fixed by excluding these index types from the libreduction path by checking against the _has_complex_internals property as I've defined it in #31238. Just a matter of merging that PR first so _has_complex_internals is available (or doing this fix as part of that PR).

[TomAugspurger]

Thanks. I think that PR can contain the check as well.

…

On Mon, Jan 27, 2020 at 9:39 AM Jeremy Schendel ***@***.***> wrote: @TomAugspurger <https://github.com/TomAugspurger> : This can be fixed by excluding these index types from the libreduction path by checking against the _has_complex_internals property as I've defined it in #31238 <#31238>. Just a matter of merging that PR first so _has_complex_internals is available (or doing this fix as part of that PR). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31248?email_source=notifications&email_token=AAKAOIQRF6V7CIARLK76CEDQ73533A5CNFSM4KKXXSK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ76NUA#issuecomment-578807504>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIXXFQJZWZCQSHO2BB3Q73533ANCNFSM4KKXXSKQ> .

[jbrockmendel]

I think @jschendel is on the right track with just avoiding _index_data altogether. For CategoricalIndex we could kludge it together, but I don't think its possible for IntervalIndex.