BUG: Timedelta.total_seconds method is returning wrong values in nanosecond intervals

[Tiarles]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

duration, Ts = 0.5, 5e-7
signal = pd.to_timedelta(np.arange(0, duration, Ts), "s")

dt = signal[1] - signal[0]

print(signal[1])  # 0 days 00:00:00.000000500
print(signal[0])  # 0 days 00:00:00

print(dt.total_seconds())  # 0.0 <- Wrong output

print(dt / pd.Timedelta(seconds=1))  # 5e-07 <- Expected behavior of ``dt.total_seconds()``

duration, Ts = 0.5, 1e-4
signal = pd.to_timedelta(np.arange(0, duration, Ts), "s")

dt = signal[1] - signal[0]

print(signal[1])  # 0 days 00:00:00.000100
print(signal[0])  # 0 days 00:00:00

print(dt.total_seconds())  # 0.0001 <- Correct answer

print(dt / pd.Timedelta(seconds=1))  # 0.0001 <- Proof

Issue Description

The total_seconds method from the Timedelta class has an unexpected behavior on Pandas 1.4.2, since Pandas 1.0.5.
For a small difference in nanoseconds scale the number of seconds on the interval returns 0,

Expected Behavior

The expected behavior is like what happens in Pandas 1.0.5, where:

import pandas as pd
import numpy as np

duration, Ts = 0.5, 5e-7
signal = pd.to_timedelta(np.arange(0, duration, Ts), "s")

dt = signal[1] - signal[0]

print(dt.total_seconds())  # = 5e-07

Installed Versions

Python 3.8.4

Env w/ Pandas 1.4.2:

Package	Version
numpy	1.22.3
pandas	1.4.2
pip	22.0.4
python-dateutil	2.8.2
pytz	2022.1
setuptools	60.10.0
six	1.16.0
wheel	0.37.1

Env w/ Pandas 1.0.5:

Package	Version
numpy	1.22.3
pandas	1.0.5
pip	22.0.4
python-dateutil	2.8.2
pytz	2022.1
setuptools	60.10.0
six	1.16.0
wheel	0.37.1

[Tiarles]

It`s an issue related to datetime package and not pandas itself. Closing

[Tiarles]

Seems that datetime.timedelta ignores nanoseconds operations that pandas should support, so the total_seconds method of pandas.core.indexes.timedeltas.TimedeltaIndex doesn't works as expected.

[simonjayhawkins]

Thanks @Tiarles for the report.

The total_seconds method from the Timedelta class has an unexpected behavior on Pandas 1.4.2, since Pandas 1.0.5.

first bad commit: [3b2c8f6] BUG: nonexistent Timestamp pre-summer/winter DST w/dateutil timezone (#31155)

see #31155 (comment) and #31354

[CloseChoice]

I don't see what we can do here, since we are relying on the CPython timedelta implementation which doesn't know nanoseconds.
The problem arises at these here:

cdef _timedelta_from_value_and_reso(int64_t value, NPY_DATETIMEUNIT reso):
    # Could make this a classmethod if/when cython supports cdef classmethods
    cdef:
        _Timedelta td_base

    if reso == NPY_FR_ns:
        td_base = _Timedelta.__new__(Timedelta, microseconds=int(value) // 1000)
    elif reso == NPY_DATETIMEUNIT.NPY_FR_us:
        td_base = _Timedelta.__new__(Timedelta, microseconds=int(value))
    elif reso == NPY_DATETIMEUNIT.NPY_FR_ms:
        td_base = _Timedelta.__new__(Timedelta, milliseconds=int(value))
    elif reso == NPY_DATETIMEUNIT.NPY_FR_s:
        td_base = _Timedelta.__new__(Timedelta, seconds=int(value))
    ...

timedelta inherits its __new__ method from CPython.timedelta which can't handle nanoseconds. Maybe trigger a warning if int(value) // 1000 == 0 and value != 0?