Skip to content

BUG: crosstab(aggfunc='skew') raises IndexError: single positional indexer is out-of-bounds #60768

Open
@sfc-gh-mvashishtha

Description

@sfc-gh-mvashishtha

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

pd.crosstab(
    index=["index1","index2"],
    columns=  ["one", "one"],
    values=[12,10],
    normalize=1,
    margins=True,
    dropna=True,
    aggfunc='skew'
)

Issue Description

The example above gives me the error IndexError: single positional indexer is out-of-bounds.

I don't get this error with aggfunc="sum".

I wonder whether this issue is related to #60767, where crosstab drops some NaN results.

stack trace
IndexError                                Traceback (most recent call last)
Cell In[2], line 3
      1 import pandas as pd
----> 3 pd.crosstab(
      4     index=["index1","index2"],
      5     columns=  ["one", "one"],
      6     values=[12,10],
      7     normalize=1,
      8     margins=True,
      9     dropna=True,
     10     aggfunc='skew'
     11 )

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/reshape/pivot.py:741, in crosstab(index, columns, values, rownames, colnames, aggfunc, margins, margins_name, dropna, normalize)
    739 # Post-process
    740 if normalize is not False:
--> 741     table = _normalize(
    742         table, normalize=normalize, margins=margins, margins_name=margins_name
    743     )
    745 table = table.rename_axis(index=rownames_mapper, axis=0)
    746 table = table.rename_axis(columns=colnames_mapper, axis=1)

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/reshape/pivot.py:783, in _normalize(table, normalize, margins, margins_name)
    781 table_index = table.index
    782 table_columns = table.columns
--> 783 last_ind_or_col = table.iloc[-1, :].name
    785 # check if margin name is not in (for MI cases) and not equal to last
    786 # index/column and save the column and index margin
    787 if (margins_name not in last_ind_or_col) & (margins_name != last_ind_or_col):

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/indexing.py:1184, in _LocationIndexer.__getitem__(self, key)
   1182     if self._is_scalar_access(key):
   1183         return self.obj._get_value(*key, takeable=self._takeable)
-> 1184     return self._getitem_tuple(key)
   1185 else:
   1186     # we by definition only have the 0th axis
   1187     axis = self.axis or 0

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/indexing.py:1690, in _iLocIndexer._getitem_tuple(self, tup)
   1689 def _getitem_tuple(self, tup: tuple):
-> 1690     tup = self._validate_tuple_indexer(tup)
   1691     with suppress(IndexingError):
   1692         return self._getitem_lowerdim(tup)

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/indexing.py:966, in _LocationIndexer._validate_tuple_indexer(self, key)
    964 for i, k in enumerate(key):
    965     try:
--> 966         self._validate_key(k, i)
    967     except ValueError as err:
    968         raise ValueError(
    969             "Location based indexing can only have "
    970             f"[{self._valid_types}] types"
    971         ) from err

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/indexing.py:1592, in _iLocIndexer._validate_key(self, key, axis)
   1590     return
   1591 elif is_integer(key):
-> 1592     self._validate_integer(key, axis)
   1593 elif isinstance(key, tuple):
   1594     # a tuple should already have been caught by this point
   1595     # so don't treat a tuple as a valid indexer
   1596     raise IndexingError("Too many indexers")

File ~/miniconda3/envs/snowpark-python-dev/lib/python3.9/site-packages/pandas/core/indexing.py:1685, in _iLocIndexer._validate_integer(self, key, axis)
   1683 len_axis = len(self.obj._get_axis(axis))
   1684 if key >= len_axis or key < -len_axis:
-> 1685     raise IndexError("single positional indexer is out-of-bounds")

Expected Behavior

I should get the skew results.

Installed Versions

INSTALLED VERSIONS
------------------
commit                : 0691c5cf90477d3503834d983f69350f250a6ff7
python                : 3.9.21
python-bits           : 64
OS                    : Darwin
OS-release            : 24.2.0
Version               : Darwin Kernel Version 24.2.0: Fri Dec  6 18:56:34 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6020
machine               : arm64
processor             : arm
byteorder             : little
LC_ALL                : None
LANG                  : en_US.UTF-8
LOCALE                : en_US.UTF-8

pandas                : 2.2.3
numpy                 : 2.0.2
pytz                  : 2024.2
dateutil              : 2.8.2
pip                   : 24.2
Cython                : None
sphinx                : None
IPython               : 8.12.0
adbc-driver-postgresql: None
adbc-driver-sqlite    : None
bs4                   : None
blosc                 : None
bottleneck            : None
dataframe-api-compat  : None
fastparquet           : None
fsspec                : None
html5lib              : None
hypothesis            : None
gcsfs                 : None
jinja2                : None
lxml.etree            : None
matplotlib            : None
numba                 : None
numexpr               : None
odfpy                 : None
openpyxl              : None
pandas_gbq            : None
psycopg2              : None
pymysql               : None
pyarrow               : None
pyreadstat            : None
pytest                : None
python-calamine       : None
pyxlsb                : None
s3fs                  : None
scipy                 : None
sqlalchemy            : None
tables                : None
tabulate              : None
xarray                : None
xlrd                  : None
xlsxwriter            : None
zstandard             : None
tzdata                : 2025.1
qtpy                  : None
pyqt5                 : None

Metadata

Metadata

Assignees

Labels

BugReshapingConcat, Merge/Join, Stack/Unstack, Explode

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions