QST: pandas.DataFrame() converts pyarrow.array() to numpy series

[yoonghm]

Research

• I have searched the [pandas] tag on StackOverflow for similar questions.

• I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

https://stackoverflow.com/questions/76648782/pandas-dataframe-converts-pyarrow-array-to-numpy-series

Question about pandas

No response

[btparrish]

from stackoverflow: As of pandas 2.0.x, the pandas constructors do not recognize pyarrow objects. In order to get a pyarrow dtype, you'll need to pass dtype=string[pyarrow]". I expect this will change in an upcoming pandas version.

[lithomas1]

I think this should work, and is a bug.

We should be preserving pyarrow dtypes if they are passed in.

cc @phofl

[mroeschke]

Just noting the current supported way for this to work is to pass your pyarrow objects to pd.arrays.ArrowExtensionArray https://pandas.pydata.org/docs/user_guide/pyarrow.html#data-structure-integration

[jbrockmendel]

The solution here is to add in sanitize_array a check for lib.is_pyarrow_array. The difficult part is ensuring that we find all the other places that may need the same check (off the top of my head pd.array)