Adapt to pandas 3.0

[tramora]

• For python 3.10, 2.3.3 is still used but with the new pandas StringDtype enabled
• For python 3.11+, the later 3.0.0+ versions are used

Fixes #536

---

TODO Before Asking for a Review

• Rebase your branch to the latest version of main (or main-v10)
• Make sure all CI workflows are green
• When adding a public feature/fix: Update the Unreleased section of CHANGELOG.md (no date)
• Self-Review: Review "Files Changed" tab and fix any problems you find
• API Docs (only if there are changes in docstrings, rst files or samples):
  • Check the docs build without warning: see the log of the API Docs workflow
  • Check that your changes render well in HTML: download the API Docs artifact and open index.html
  • If there are any problems it is faster to iterate by building locally the API Docs

[popescu-v]

See the remaining comments.

[popescu-v]

In [1]: import pandas as pd
In [2]: pd.__version__
Out[2]: '3.0.0'
In [3]: x = pd.DataFrame(["a", "b", "c"]).astype(pd.StringDtype())
In [4]: x[0].str.len().max()
Out[4]: np.int64(1)

Hence, it seems than (on Pandas 3.0.0), .str.len() returns a numpy.int64.

[tramora]

But unfortunately Series.str.len() returns a float64 and the previous code broke a specific test
https://pandas.pydata.org/docs/reference/api/pandas.Series.str.len.html#pandas.Series.str.len

[popescu-v]

This is only the case if at least one element is not of string type, in which case .str.len() results in a NaN:

In [1]: y = pd.Series(["a", 5, "c"])

In [2]: y.str.len()
Out[2]: 
0    1.0
1    NaN
2    1.0
dtype: float64

But this is no longer the case if all elements are properly typed to string:

In [3]: y = pd.Series(["a", 5, "c"]).astype(pd.StringDtype())

In [4]: y.str.len()
Out[4]: 
0    1
1    1
2    1
dtype: Int64

However, the code executed here is in the scope of an if isinstance(column.dtype, pd.StringDtype); hence, the return type should be Int64 AFAIU.

[tramora]

I reverted to the previous version of the code, if a test break I'll have a look at the exact condition (if a NaN remains in the series)

[popescu-v]

It breaks, because there are missing (StringDtype) values in the input dataframe. See also pandas-dev/pandas#51948. Hence, I would:

• add a comment before this line of code, stating the issue and referring to the aforementioned issue
• cast to a Pandas integer data type: columns.str.len().astype(pd.Int64Dtype()).max().

[tramora]

I commented and added the astype conversion call

[folmos-at-orange]

LGTM.

I would change the README for the minimum versions though (or eliminate that section).

[popescu-v]

One single remaining pending change (see the comment).