REGR: passing dask arrays to Series or DataFrame

[keewis]

Code Sample, a copy-pastable example

import pandas as pd
import dask.array as da
a = da.ones((12,), chunks=4)
s = pd.Series(a, index=range(12))
print(s.dtype)

Problem description

This has been detected by xarray's upstream-dev CI (environment): with 1.1.3, the dtype is float64 while on master (installed from scipy-wheels-nightly) this became object (and the series / dataframe contains dask scalars). Was that change intentional? Poking around on the merged PR list, this might have been #38563 (not sure, though).

To be clear, for us this only affects test code and since it would compute anyways we can easily work around this by computing the dask array before passing it to pd.Series or pd.DataFrame.

See also pydata/xarray#4717.

cc @TomAugspurger

[jorisvandenbossche]

@keewis thanks for the report! Can confirm the change in behaviour.

cc @jbrockmendel

[simonjayhawkins]

Poking around on the merged PR list, this might have been #38563 (not sure, though).

can confirm, first bad commit: [cec2f5f] REF: handle non-list_like cases upfront in sanitize_array (#38563)

[jbrockmendel]

the low-level place to fix this would be in is_list_like, question is if we can do that without a big performance hit

[jbrockmendel]

One way to fix on dask's end would to be to implement Array.__iter__. Is that a viable option?

[keewis]

I'm going to forward this to the dask devs: cc @TomAugspurger, @jsignell, @jrbourbeau