ENH: implement DateOffset + np.datetime64 array

[hottwaj]

Is your feature request related to a problem?

DateOffset + a numpy array of type datetime64 does not work, see examples at the end of this post

Describe the solution you'd like

can __add__ or __radd__ of DateOffset be changed to implement addition to datetime64 array?

API breaking implications

I am not the right person to comment, but seems unlikely to have an impact on existing API if it can be done at all

Describe alternatives you've considered

Sorry, not thought of alternatives so far

Code example

>>> import pandas
>>> import numpy
>>> dates = pandas.Series([pandas.Timestamp(2021, 1, 1) + pandas.Timedelta(days=d) for d in range(1000)])
>>> offset = pandas.offsets.MonthEnd()

>>> # adding a pandas Timestamp series to an offset obviously works
>>> dates + offset
0     2021-01-31
1     2021-01-31
         ...    
998   2023-09-30
999   2023-09-30
Length: 1000, dtype: datetime64[ns]

>>> # adding the underlying numpy datetime64 values does not work 
>>> offset + dates.values
---------------------------------------------------------------------------
UFuncTypeError                            Traceback (most recent call last)
----> 1 offset + dates.values

UFuncTypeError: ufunc 'add' cannot use operands with types dtype('O') and dtype('<M8[ns]')

>>> # slightly different error if we swap the order of the two variables
>>> dates.values + offset
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
----> 1 dates.values + offset

TypeError: Concatenation operation is not implemented for NumPy arrays, use np.concatenate() instead. Please do not rely on this error; it may not be given on all Python implementations.

>>> # the desired result can be achieved by using DateOffset._apply_array() and casting to M8[ns]
>>> offset._apply_array(dates.values).astype('M8[ns]')
array(['2021-01-31T00:00:00.000000000', '2021-01-31T00:00:00.000000000',
       '2021-01-31T00:00:00.000000000', '2021-01-31T00:00:00.000000000',
       '2021-01-31T00:00:00.000000000', '2021-01-31T00:00:00.000000000',
       '2021-01-31T00:00:00.000000000', '2021-01-31T00:00:00.000000000',
       etc])

[hottwaj]

From a bit of testing and looking at numpy docs, adding a __array_ufunc__ method to DateOffset class seems like a possible route to make this work

Relevant numpy docs here:
https://numpy.org/doc/stable/user/basics.subclassing.html#array-ufunc-for-ufuncs

Example class that can wrap any offset and allow it to be added to numpy.ndarray below. I expect some more checks on e.g. dtype might be needed to help catch incorrect usage (e.g. offset + ndarray of int)

class OffsetWrapper:
    def __init__(self, offset):
        self.offset = offset
        
    def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
        a, b = inputs
        if isinstance(a, numpy.ndarray) and isinstance(b, OffsetWrapper):
            return b.offset._apply_array(a).astype('M8[ns]')
        elif isinstance(b, numpy.ndarray) and isinstance(a, OffsetWrapper):
            return a.offset._apply_array(b).astype('M8[ns]')
        else:
            return NotImplemented

Using that class, both of these will work

dates.values + OffsetWrapper(offset)
OffsetWrapper(offset) + dates.values

So I think adding array_ufunc to DateOffset could make this work?

Will assemble into a PR, but please let me know if anyone foresees any issues with this approach. Thanks!

[jbrockmendel]

The relevant logic lives in Offset._apply_array. Would need to update Offset.__add__ to dispatch where relevant.

[hottwaj]

Thanks for comments above.

I have come back to this after a long time and added an initial PR here: #57927

As noted there, to progress I need some guidance on:

• cython mechanism to check that passed ndarray is datetime64 dtype i.e.cython equivalent of pandas.api.types.is_datetime64_any_dtype
• how to approach adding unit tests? two ideas I have
  • somehow duplicate existing offset tests but run them on ndarray instead of Series? looks on the surface to be more difficult to implement
  • apply a set of offsets to a predefined Series and ndarray equivalent, and then check results match

Many thanks!