df.info() ignores index hashtable memory usage

[kay1793]

another problem with memory report (#7619) in df.info() like #8578 (locked) and fix to multindex hidden in #8456 (see whatsnew). it count the space used by index hashtable. for each entry the klib hashtable keeps a copy of the index data as well as a key. for strings that is a copy of the string + pointer. in worst case there is 2*next_power_of_2(len(index)) entries and problem exists whatever dtype the index use.

x.index._engine.mapping
Out[13]: <pandas.hashtable.PyObjectHashTable at 0x7fb92f32bef0>

cc @shoyer, @njsmith
cc @asobrien

[jreback]

@kay1793 not likely to get this for quite a while. you are welcome to submit a fix. Keep in mind that this is quite complicated as index object implementations are shared among same indexes.

[kay1793]

I understand.

[TomAugspurger]

@jreback was this closed by #15237?

[jreback]

yep