Skip to content

Indexing on series where index is output from pd.cut  #27437

Open
@kuchenrolle

Description

@kuchenrolle

Code Sample, a copy-pastable example if possible

import pandas as pd

data = pd.DataFrame(dict(values=range(20), quintiles=pd.cut(range(20), 5)))
grouped = data.groupby("quintiles")["values"]
samples = grouped.agg(lambda x: list(x.sample(3)))

samples
# quintiles
# (-0.019, 3.8]       [3, 2, 1]
# (3.8, 7.6]          [7, 4, 6]
# (7.6, 11.4]       [8, 10, 11]
# (11.4, 15.2]     [15, 14, 12]
# (15.2, 19.0]     [17, 16, 19]
# Name: values, dtype: object

samples[0]
# [3, 2, 1]
samples[1]
# [3, 2, 1]
samples[2.3]
# [3, 2, 1]
samples[19]
# [17, 16, 19]

samples.keys()
# CategoricalIndex([(-0.019, 3.8], (3.8, 7.6], (7.6, 11.4], (11.4, 15.2],
#                   (15.2, 19.0]],
#                  categories=[(-0.019, 3.8], (3.8, 7.6], (7.6, 11.4], (11.4, 15.2], (15.2, 19.0]], ordered=True, name='quintiles', dtype='category')

Problem description

Indexing a series object that has pd.cut() intervals as its index results in membership test of the provided index on the intervals, which I found interesting, but it was very unexpected. Is this intentional and documented somewhere? When I give the labels names (labels=list("abcde") in pd.cut), this doesn't happen anymore.

Expected Output

I expectived it to fall back on .iloc[].

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.7.3.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-957.12.2.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.24.2
pytest: 4.4.1
pip: 19.1.1
setuptools: 41.0.1
Cython: 0.29.6
numpy: 1.16.3
scipy: 1.2.1
pyarrow: None
xarray: 0.12.1
IPython: 7.4.0
sphinx: None
patsy: 0.5.1
dateutil: 2.8.0
pytz: 2019.1
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.1.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: 4.3.3
bs4: 4.7.1
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

Metadata

Metadata

Assignees

Labels

IndexingRelated to indexing on series/frames, not to indexes themselvesNeeds TestsUnit test(s) needed to prevent regressionsgood first issue

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions