Enable label-based indexing with .loc even when DataFrame is not lexsorted

[dr-leo]

As shown in the following code example, .loc is more restrictive than the deprecated .ix. Label-based indexing without slicing should be possible with .loc even when the dataframe is not lexsorted. The docs could be clearer in this respect. In addition, performance warnings should ideally be suppressible through options.

In [1]: from pandasdmx import *

In [2]: estat = Request('estat')

In [3]: dsd_resp = estat.datastructure('DSD_une_rt_a')

In [4]: df = dsd_resp.write().codelist

In [5]: df.ix[['AGE', 'UNIT']]
C:\Users\stefan\Anaconda3\envs\py35\Scripts\ipython-script.py:1:
DeprecationWarn
ing:
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate_ix

if name == 'main':
Out[5]:
dim_or_attr name
AGE AGE D AGE
TOTAL D Total
Y25-74 D From 25 to 74 years
Y_LT25 D Less than 25 years
UNIT UNIT D UNIT
PC_ACT D Percentage of active population
PC_POP D Percentage of total population
THS_PER D Thousand persons

In [6]: df.loc[['AGE', 'UNIT']]

UnsortedIndexError Traceback (most recent call last)
in ()
----> 1 df.loc[['AGE', 'UNIT']]

C:\Users\stefan\Anaconda3\envs\py35\lib\site-packages\pandas\core\indexing.py
in
getitem(self, key)
1326 else:
1327 key = com._apply_if_callable(key, self.obj)
-> 1328 return self._getitem_axis(key, axis=0)
1329
1330 def _is_scalar_access(self, key):

C:\Users\stefan\Anaconda3\envs\py35\lib\site-packages\pandas\core\indexing.py
in
_getitem_axis(self, key, axis)
1543 # nested tuple slicing
1544 if is_nested_tuple(key, labels):
-> 1545 locs = labels.get_locs(key)
1546 indexer = [slice(None)] * self.ndim
1547 indexer[axis] = locs

C:\Users\stefan\Anaconda3\envs\py35\lib\site-packages\pandas\core\indexes\multi.
py in get_locs(self, tup)
2267 'to be fully lexsorted
tuple len ({
0}), '
2268 'lexsort depth ({1})'
-> 2269 .format(len(tup),
self.lexsort_dept
h))
2270
2271 # indexer

UnsortedIndexError: 'MultiIndex Slicing requires the index to be fully
lexsorted
tuple len (1), lexsort depth (0)'

I'd like to see .loc do the job as .ix did, maybe with a performance
warning. Ideally the latter should be suppressible by setting an option
rather than wrapping everything in a context manager from the warnings
stdlib module.

[toobaz]

Label-based indexing without slicing should be possible with .loc even when the dataframe is not lexsorted

Notice this is specific to indexing with lists (and booleans, but not single labels). Anyway, I'll push a PR in minutes.

[dr-leo]

Great! To add one perhaps self-evident corner case: in a multi-indexed df, ':' should be permissible as well. df.loc[['a', 'c'], :, ['two', 'one']] This is not a tested code example. But the idea should be clear: If the slice captures the entire row/col, sorting should not be required either. Am 20.06.2017 um 11:03 schrieb Pietro Battiston:

…

Label-based indexing without slicing should be possible with .loc even when the dataframe is not lexsorted Notice this is specific to indexing with lists (and booleans, but not single labels). Anyway, I'll push a PR in minutes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#16734 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADA58tH0g-2wYxOZpP_7dFMyrQ35Nzbxks5sF4r4gaJpZM4N_QR5>.

[toobaz]

If the slice captures the entire row/col, sorting should not be required either.

Yes, the fix contemplates this use case.

[jreback]

well @toobaz pushed a fix, but ideally you would have a copy-pastable example in the top section. IOW is just code and not relying on external library.