[Python] Conversion to/from numpy 2.0+ new StringDType

[jorisvandenbossche]

With the upcoming numpy 2.0 release, they introduce a new variable-length string dtype, StringDType (https://numpy.org/devdocs/release/2.0.0-notes.html#highlights).

Currently trying to convert that obviously gives an error:

>>> arr = np.array(["some", "strings"], dtype=np.dtypes.StringDType())
>>> arr
array(['some', 'strings'], dtype=StringDType())
>>> pa.array(arr)
...
ArrowNotImplementedError: Unsupported numpy type 2056

But ideally we should support this dtype both for conversion from numpy -> arrow, as in the arrow -> numpy conversion (although here it should probably be opt-in and by default still return object dtype)

Numpy provides a C API to access the individual string elements: https://numpy.org/devdocs/reference/c-api/strings.html

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.

[alippai]

I believe this is not stale, numpy is continuously improving the StringDType support.

@jorisvandenbossche would this be implemented on the C++ side or cython/python?