-
-
Notifications
You must be signed in to change notification settings - Fork 19.3k
PERF: define is_all_dates to shortcut inadvertent copy when slicing an IntervalIndex #23591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Hello @qwhelan! Thanks for submitting the PR.
|
Codecov Report
@@ Coverage Diff @@
## master #23591 +/- ##
==========================================
+ Coverage 92.25% 92.25% +<.01%
==========================================
Files 161 161
Lines 51237 51239 +2
==========================================
+ Hits 47269 47271 +2
Misses 3968 3968
Continue to review full report at Codecov.
|
| return func | ||
|
|
||
| @property | ||
| def is_all_dates(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note that we should actually change the default for Index suclasses i think
|
Thanks @TomAugspurger ! |
We get a few orders of magnitude speedup in
IntervalIndexslicing by simply overriding the base class definition ofis_all_dates, like all otherIndexderivatives also do. The root cause of the performance degradation is as follows:Series, a newSeriesis created for the result.Series.__init__()isSeries._set_axis(), which in turn calls.is_all_dateson the newIndexIndex.is_all_datesis:which seems harmless at first glance. However, this eventually invokes
IntervalArray.__array__, which is a pure Python for-loop creatingIntervalobjects and leading to the performance regression here.As the value of
IntervalIndex.is_all_datesappears to always beFalse, even in the case of datetime-like left/right values, we simply override to return that value and shortcut the inadvertent copy described above.Benchmarks
Speed up of ~10704x for
time_loc_listgit diff upstream/master -u -- "*.py" | flake8 --diff