BUG: Pandas 2.0 loses type information from datetime64[M/D] when creating dataframes

[KianShepherd]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np


a = {
    'x': np.array([18993, 19000, 19045], dtype=np.dtype('datetime64[D]')),
    'y': np.array([18993, 19000, 19045], dtype=np.dtype('datetime64[M]'))
}

print(f'{[x.dtype for x in a.values()]}\n')
print(pd.DataFrame(data=a).dtypes)

Output:

[dtype('<M8[D]'), dtype('<M8[M]')]

x    datetime64[s]
y    datetime64[s]
dtype: object

Issue Description

Using pandas 1.x creating a dataframe using datetime64[D] or datetime64[M] types would maintain the type information within the resulting columns. However when using pandas 2.x these arrays are now both converted to datetime64[s]. This loses track of the fact that initially the inputs were typed as days and months.

Expected Behavior

When using pandas 2.x I would expect that the type information of months/days to be preserved or at a minimum that there be some property that can be used to determine if a datetime64[s] is actually a month or day.

Installed Versions

INSTALLED VERSIONS

commit : e86ed37
python : 3.10.13.final.0
python-bits : 64
OS : Linux
OS-release : 6.5.3-arch1-1
Version : #1 SMP PREEMPT_DYNAMIC Wed, 13 Sep 2023 08:37:40 +0000
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.1.1
numpy : 1.25.2
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 65.5.0
pip : 23.2.1
Cython : 3.0.0a11
pytest : 7.1.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.15.0
pandas_datareader : None
bs4 : 4.12.2
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 9.0.0
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

[mroeschke]

Thanks for the report. Per the 2.0 release notes day and month resolution are not explicitly not supported: https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html#construction-with-datetime64-or-timedelta64-dtype-with-unsupported-resolution

However the silent coercion of a dict of unsupported resolution arrays should probably raise instead of coercing to seconds, so a PR to do that would be welcome!

[KianShepherd]

@mroeschke Thanks for replying, I'll try and look into making an MR in the future to add that. Is there anything internally that can be used to determine if it is a day / month? or is that type information just completely lost?

[mroeschke]

Unfortunately there's no way to retain that information in pandas; it wasn't really supported/meaningful in pandas 1.x too.

The closest equivalent is using pd.infer_freq to assign a frequency to a DatetimeIndex, but the data spacing must conform to that.

[jbrockmendel]

if converting to M8[s] is a deal-breaker, id usually convert to period[D] or period[M] which quack like the numpy D/M dtypes.

[jorisvandenbossche]

However the silent coercion of a dict of unsupported resolution arrays should probably raise instead of coercing to seconds, so a PR to do that would be welcome!

We have always coerced datetime64 arrays with unsupported resolutions to something we support (historically only ns), so I am not sure we should stop doing that.