Skip to content

REF: Localizer class to de-duplicate tzconversion code #46397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Apr 18, 2022
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
40dcda5
Implement Localizer - perf neutral
jbrockmendel Mar 7, 2022
db075f9
perf-neutral-ish
jbrockmendel Mar 7, 2022
dde49a6
move Localizer to top
jbrockmendel Mar 7, 2022
b1ba933
move delta back inline
jbrockmendel Mar 8, 2022
f5e0503
troubleshoot perf
jbrockmendel Mar 9, 2022
45e5685
remove initializedcheck(False)
jbrockmendel Mar 10, 2022
5a66852
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 10, 2022
828d6fc
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 11, 2022
53cd93d
fewer lines
jbrockmendel Mar 11, 2022
22d4aac
troubleshoot perf
jbrockmendel Mar 11, 2022
492897e
troubleshoot perf
jbrockmendel Mar 11, 2022
07b934f
remove unused declarations
jbrockmendel Mar 11, 2022
605100f
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 12, 2022
70e54d7
lint
jbrockmendel Mar 12, 2022
da32448
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 12, 2022
3c064b6
CLN, avoid build warning
jbrockmendel Mar 12, 2022
c0b467f
BUG: replace with value also being replaced (#46335)
lukemanley Mar 12, 2022
6334376
REF: de-duplicate libjoin (#46256)
jbrockmendel Mar 12, 2022
eddacb3
use bisect_right_i8
jbrockmendel Mar 13, 2022
b342bbd
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 14, 2022
6b6292d
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 16, 2022
922fcb7
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 16, 2022
c1085df
micro-optimize?
jbrockmendel Mar 16, 2022
33e893e
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 18, 2022
ebb5370
troubleshoot
jbrockmendel Mar 18, 2022
41f89cf
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 20, 2022
11e3e70
troubleshoot
jbrockmendel Mar 20, 2022
a46d127
inline info.delta
jbrockmendel Mar 20, 2022
c479050
revert troubleshooting
jbrockmendel Mar 20, 2022
b4427a5
troubleshoot
jbrockmendel Mar 20, 2022
1b620b9
troubleshoot
jbrockmendel Mar 20, 2022
a1b3e00
troubleshoot
jbrockmendel Mar 20, 2022
09dba27
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 20, 2022
6b36911
Merge branch 'main' into ref-localizer-2
jbrockmendel Mar 29, 2022
a5c5142
try a thing
jbrockmendel Mar 29, 2022
76132b6
woops
jbrockmendel Mar 29, 2022
ba9f3a8
Merge branch 'main' into ref-localizer-2
jbrockmendel Apr 12, 2022
63defc7
Merge branch 'main' into ref-localizer-2
jbrockmendel Apr 13, 2022
6d39fee
Merge branch 'main' into ref-localizer-2
jbrockmendel Apr 15, 2022
1bf0898
Merge branch 'main' into ref-localizer-2
jbrockmendel Apr 18, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
troubleshoot
  • Loading branch information
jbrockmendel committed Mar 20, 2022
commit 11e3e701f615db40936bd8ab4b3b4df2c9f8fe07
15 changes: 12 additions & 3 deletions pandas/_libs/tslibs/vectorized.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,14 @@ def get_resolution(const int64_t[:] stamps, tzinfo tz=None) -> Resolution:

# -------------------------------------------------------------------------

'''
When using info.use_utc checks inside the loop...
+ 2.40±0.06ms 3.96±0.2ms 1.65 tslibs.normalize.Normalize.time_normalize_i8_timestamps(1000000, datetime.timezone(datetime.timedelta(seconds=3600)))
+ 30.6±2μs 48.3±1μs 1.58 tslibs.normalize.Normalize.time_normalize_i8_timestamps(10000, datetime.timezone(datetime.timedelta(seconds=3600)))
+ 3.56±0.08ms 5.24±0.4ms 1.47 tslibs.normalize.Normalize.time_normalize_i8_timestamps(1000000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
+ 39.5±2μs 57.5±3μs 1.46 tslibs.normalize.Normalize.time_normalize_i8_timestamps(10000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
'''

@cython.wraparound(False)
@cython.boundscheck(False)
cpdef ndarray[int64_t] normalize_i8_timestamps(const int64_t[:] stamps, tzinfo tz):
Expand All @@ -283,6 +291,7 @@ cpdef ndarray[int64_t] normalize_i8_timestamps(const int64_t[:] stamps, tzinfo t
int64_t utc_val, local_val
Py_ssize_t pos, i, n = stamps.shape[0]
int64_t* tdata = NULL
bint use_utc=info.use_utc,use_tzlocal=info.use_tzlocal,use_fixed=info.use_fixed

int64_t[::1] result = np.empty(n, dtype=np.int64)

Expand All @@ -295,11 +304,11 @@ cpdef ndarray[int64_t] normalize_i8_timestamps(const int64_t[:] stamps, tzinfo t
result[i] = NPY_NAT
continue

if info.use_utc:
if use_utc:
local_val = utc_val
elif info.use_tzlocal:
elif use_tzlocal:
local_val = utc_val + localize_tzinfo_api(utc_val, tz)
elif info.use_fixed:
elif use_fixed:
local_val = utc_val + info.delta
else:
pos = bisect_right_i8(tdata, utc_val, info.ntrans) - 1
Expand Down