TypeError when datetime column is all null (Cannot cast DatetimeArray to dtype float64)

[npatki]

Environment Details

• SDV version: 1.2.0
• Python version: Any
• Operating System: Any

Error Description

Sometimes I may have a datetime column that contains all NULL values (see RDT #367 for why this may happen).

If I try to input this column into the SDV, then I get an error when sampling.

Steps to reproduce

import pandas as pd
from sdv.metadata import SingleTableMetadata
from sdv.single_table import GaussianCopulaSynthesizer

metadata_dict = {
    'columns': {
        'A': { 'sdtype': 'datetime', 'datetime_format': '%Y-%m-%d'},
        'B': { 'sdtype': 'numerical' }
    }
}

metadata = SingleTableMetadata.load_from_dict(metadata_dict)
data_nulls = pd.DataFrame(data={
    'A': [np.nan, np.nan, np.nan, np.nan, np.nan],
    'B': [23.12, 59.12, 10.00, 12.01, 10.11]
})

synth = GaussianCopulaSynthesizer(metadata)
synth.fit(data_nulls)
synth.sample(5)

Output:

TypeError: Cannot cast DatetimeArray to dtype float64

Additional Context

• The sampling works if the column contains all numerical values
• The _sample method returns a column full of 0.0 values (as intended)
• The RDT HyperTransformer works fine for forward and reverse transform (see below). My guess is that since the values are reversed to pd.NaT, the SDV is erroring because it wants to convert them into floats somehow?

[npatki]

Root Cause: Even though the input data has NaN values, the RDT is providing NaT values during sampling. This will be fixed by RDT #657

Workaround

For now, you can workaround this issue by converting the input data to NaT values instead of leaving them as NaN.

import pandas as pd

# this converts the NaN values to NaT values
data_nulls['A'] = pd.to_datetime(data_nulls['A'])

# now you can model and sample
synth.fit(data_nulls)
synth.sample(5)

[npatki]

This has been fixed.