Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
from datetime import datetime
tznow = datetime.now().astimezone()
print(tznow.isoformat())
now = datetime.now()
i = pd.date_range(end=now, periods=4, freq='1H')
ts = pd.DataFrame({'A': [1, 2, 3, 4]}, index=i)
print()
print('all')
print(ts)
print()
print('3H naive')
print(ts.last('3H'))
i_zone = pd.date_range(end=now, periods=4,
freq='1H').tz_localize('America/Phoenix')
ts_zone = pd.DataFrame({'A': [1, 2, 3, 4]}, index=i_zone)
print()
print('all zone')
print(ts_zone)
print()
print('3H zone')
print(ts_zone.last('3H'))Issue Description
At time of running tznow prints as: 2023-03-23T01:11:48.622675+00:00, aka 1:11 am on 23rd
We see the example for the docs with no time zone offset works correctly:
all
A
2023-03-22 22:11:48.622795 1
2023-03-22 23:11:48.622795 2
2023-03-23 00:11:48.622795 3
2023-03-23 01:11:48.622795 4
3H naive
A
2023-03-22 23:11:48.622795 2
2023-03-23 00:11:48.622795 3
2023-03-23 01:11:48.622795 4
The 22:11 time is omitted, because 22:11 on 22nd is more than 3 hours before aka 01:11 on 23rd ✅
Now we repeat but use tz_localize to assert that the times are all 7 hours behind UTC:
all zone
A
2023-03-22 22:11:48.622795-07:00 1
2023-03-22 23:11:48.622795-07:00 2
2023-03-23 00:11:48.622795-07:00 3
2023-03-23 01:11:48.622795-07:00 4
In UTC these would be 15:11, 16:11, 17:11, and 18:11 (and all on 22nd).
These times are all more than 3 hours behind the current time: 1:11 am on 23rd.
However the same three rows are erroneously returned:
3H zone
A
2023-03-22 23:11:48.622795-07:00 2
2023-03-23 00:11:48.622795-07:00 3
2023-03-23 01:11:48.622795-07:00 4
Expected Behavior
print(ts_zone.last('3H')) outputs zero rows, since none times are within the last 3 hours.
Installed Versions
INSTALLED VERSIONS
commit : 2e218d1
python : 3.10.8.final.0
python-bits : 64
OS : Linux
OS-release : 5.15.0-1030-gcp
Version : #37~20.04.1-Ubuntu SMP Mon Feb 20 04:30:57 UTC 2023
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.5.3
numpy : 1.24.1
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 63.2.0
pip : 21.2.dev0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None