Have convert_dtypes() function for dataframes

[leoliu0]

Description

It will be great to have pandas' df.convert_dtypes() that cast columns to the best possible dtypes.

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.convert_dtypes.html

This will convert, for example, all float columns that are actually integers like [1.0, 2.0, 3.0] to [1, 2, 3].

This could be a option in shrink_to_fit()

[deanm0000]

You say, "for example" convert Floats to Ints but it's not clear if that's the only feature you're looking for that isn't already in shrink_to_fit.

One issue I see, 2^32 can be stored as a Float32 but not as a UInt32 so I guess it would need two checks, one that it is properly rounded and the other that it isn't bigger than the max int size. That said, it should probably not really use UInts so it'd really be 2**32/2-1 as the highest.

Another issue is floating point error. For example:

pl.select(a=pl.lit(2**32/2-2,pl.Float32)+0.5).with_columns(is_rounded=pl.col("a")==pl.col("a").round())
shape: (1, 2)
┌──────────┬────────────┐
│ a        ┆ is_rounded │
│ ---      ┆ ---        │
│ f32      ┆ bool       │
╞══════════╪════════════╡
│ 2.1475e9 ┆ true       │
└──────────┴────────────┘

It thinks that 2**32/2-1.5 is already rounded because Float32 can't manage the precision. As the 0.5 gets to be more like 0.01 or 0.000001, then that gets even worse.

There'd need to be a cutoff that is much lower than the apparent integer size limit that it wouldn't attempt a conversion. For example, maybe for Float32 it only considers up to 27 and for Float64 only considers up to 236 to help mitigate the floating point error in checking if a number can become an integer.

[leoliu0]

Thank you. I think there are only two feature needed: Convert floating points like 1.0 to 1, and datetime which are actually dates like 12:00:00 00:00:00 to 12:00:00.

I cannot think of more stuff needs to be converted.