BUG: Using tuples to specify single columns in .loc

[SudhanAnnamalai]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randn(5,4), 
                  index = ["a", "b", "c", "d", "e"],
                  columns = ["One", "Two", "Three", (6, "two")])

Issue Description

#Error line
df.loc[:, (6, "two")] #KeyError: "None of [Index([6, 'two'], dtype='object')] are in the [columns]"
df.loc["a", (6, "two")] # Assertion Error : assert retval.ndim == self.ndim

Expected Behavior

However, the code below works fine(as expected)

1. df.loc["a", "Two"]
2. df.loc[:, "Two"]
3. df.loc[:, [(6, "two")]]
4. df.loc["a", [(6, "two")]]

Either, the syntax should work regardless of the datatype of the column_name, like in our example : tuple, or else the module should raise a exception or error with a clear information saying that the tuple based column name has to be changed.

I have also checked the above scenario, with other methods.
Providing a single or multiple tuple based columne(s) results a data frame without error as in the above (3) (4)
That could recommended along with the error statement, This could save a lot of time to the end user

Installed Versions

INSTALLED VERSIONS

commit : 2e218d1
python : 3.10.1.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 1.5.3
numpy : 1.24.0
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 58.1.0
pip : 21.2.4
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.2
html5lib : None
pymysql : None
psycopg2 : 2.9.6
jinja2 : 3.1.2
IPython : 8.12.0
pandas_datareader: None
bs4 : 4.12.2
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.6.2
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.10.0
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : 2022.7

[JostBrand]

Non single item nested Lists and Tuples are unpacked and matched individually, which is on purpose. Imagine the following example in which you want to specify which information to return. Both calls work but return different dataframes.

import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randn(5, 4), 
                  index = ["a", "b", "c", "d", "e"],
                  columns = [6, "two", "Three", (6, "two")])

df.loc[:, (6, "two")]
df.loc[:,[(6, "two")]]

Output 1:

	6	two
a	-1.84911	-1.58843
b	-1.258	0.433662
c	-0.401985	-0.654233
d	0.670789	0.92318
e	1.48344	0.683083

Output 2:

	(6, 'two')
a	1.09665
b	-0.709204
c	-0.174068
d	0.379871
e	0.0531941

[SudhanAnnamalai]

I totally agree with your point, but why would we unpack the tuple as columns while we can do that with the list based approach. Doesn't it create a confusion here?

Consider there is a big project which used** .loc** function to slice a part of data in which the column name is tuple based, and it throws error as I mentioned in the issue.

Do you think it's easy to debug the code with the error and exception raises.

Either the code should raise a appropriate exception or it should enforce a rule to restrict a user from using tuple based column names.

I am open to discuss a logical and valid points, and correct me if there is any other perspectives.

And I don't see a documentation which demands this squared bracket enclosed format while I want to fetch a single column, where the column name is a tuple.

[rhshadrach]

@SudhanAnnamalai - can you give this issue a descriptive title?

[SudhanAnnamalai]

@SudhanAnnamalai - can you give this issue a descriptive title?

Sure

[rhshadrach]

pandas does at times treat tuples and lists differently, and at first glance this seems to be a case where perhaps we should do so here. However it might be more complicated - for example, tuples are also used to indicate levels of a MultiIndex - so there is a number of cases to consider here.

cc @pandas-dev/pandas-core for any thoughts.

[attack68]

There are many cases to consider: see this thread (which may be a bit out of date in operation terms but not in function): #39424 (comment)

That specific link is a pathological example of multindexes where tuples form part of an element in a multiindex. Allowing tuples both as element labels and as containers for multinindex element labels in my opinion creates a plethora of problems without really giving a lot to users. Users could probably redefine their element tuples as something else.

@jbrockmendel proposed something else that didnt get much traction and didnt stir a discussion, unfortunately. #42349