BUG: Series/Index results in datetime/timedelta incorrectly if inputs are all nan/nat like

[sinhrks]

• closes API: NaT/nan mixed input coerces to timedelta64 #13467
• tests added / passed
• passes git diff upstream/master | flake8 --diff

[codecov-io]

Current coverage is 84.33%

Merging #13477 into master will not change coverage

@@             master     #13477   diff @@
==========================================
  Files           138        138          
  Lines         51092      51092          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43087      43087          
  Misses         8005       8005          
  Partials          0          0          

Powered by Codecov. Last updated by c2cc68d...a017a8c

[jreback]

I think I might move this here as well (though may cause a circular import issue).

[jreback]

@sinhrks puzzled why this fixed things as all you did was move something

[sinhrks]

ATM, it doesn't affect to the behavior. However, I found some inconsistency when I adding the test. Updated in #13467.

[jreback]

if its the same logic, then let's just add a flag that includes/excludes it rather then replicating the logic
(or make another function sitting below it that does that).

[sinhrks]

@jreback The logic should be updated after a decision in #13467. Could you check?

[jreback]

ok as I commented on the other issue, I think all -> NaT/nan should be M8.

[jreback]

@sinhrks ready to merge? or is there still an open question on this

[sinhrks]

@jreback Updated based on #13467, and found that there is a test which expect np.timedelta('nat') is regarded as timedelta dtype (I think it's natural)

• https://github.com/pydata/pandas/blob/master/pandas/tests/series/test_quantile.py#L42

How about skipping only np.nan and pd.NaT in infer_dtype? I mean:

• [pd.NaT, np.datetime64('nat')] -> datetime
• [pd.NaT, np.timedelta64('nat')] -> timedelta
• [np.datetime64('nat'), np.timedelta64('nat')] -> mixed (object)

[jreback]

yeah that looks right

the problem is we can't know definitively the inferred type if we only have pd.NaT (or just np.nan for that matter)

so I suppose object is correct in those cases (though might break things)

[sinhrks]

Yeah. pd.NaT should be ragarded as datetime in all nan-like case, if not specified otherwise (no np.datetime or np.timedelta).

• [pd.NaT, np.nan] -> datetime (it results in timedelta on current master, NG)
• [np.nan, pd.NaT] -> datetime (OK on current master)
• [pd.NaT, pd.NaT] -> datetime

[sinhrks]

Updated based on the above discussion.

[jreback]

hmm I thought this was needed maybe on py3? (the integer object check and then check for timedelta)

[sinhrks]

Though test has been passed, will revert it for safety.

[jreback]

do we need to export these (as in tslib.pxd)?

[sinhrks]

Not needed ATM because tslib doesn't use is_null_datetimelike.

[jreback]

@sinhrks looks really good. esp all of the added tests! just a couple of very minor questions. ping when ready.

[jorisvandenbossche]

What is the difference with the exp above (exp = Series([pd.NaT, pd.NaT]))?

[sinhrks]

Not needed because of the bug fix. Removed.

[sinhrks]

Now all green.

[sinhrks]

This must be moved here. Otherwise, timedelta and object mixed data is regarded as "mixed-integer".

[jreback]

IIRC this was green. but rebase and ping.

[sinhrks]

@jreback @jorisvandenbossche rebased, and all green.

[jorisvandenbossche]

@sinhrks Thanks!