-
Notifications
You must be signed in to change notification settings - Fork 301
Description
🐛 Bug Report
iris.pandas._get_dimensional_metadata requires improvements to test-coverage and code to handle scalar date-time columns during the cube conversion process.
This was noticed during #6948:
@trexfeathers Would appreciate your help in tracking down this error that relates to this call failing at https://github.com/hsteptoe/iris/blob/fc297b6ef463f3492832339b14ce691a03ddc710/lib/iris/pandas.py#L69.
Maybe I have typed
valuesincorrectly, but it seems a possible bug thatvaluescan be passed something that isn't iterable??The original code did indeed contain a bug, and you've made a mistake too.
Original bug
This code would never work when presented with a scalar time column, for multiple reasons (below). We never noticed because there simply isn't test coverage for scalar time columns, only scalar numeric columns.
Checking
values.dtype
valueswill not always have adtypewhen it is scalar, due to how NumPy handlesobjectarrays (such as arrays of date-times`):
array.dtypearray[0]hasattr(array[0], "dtype")recognised e.g. float32/boolNumPy scalar e.g. np.float32(2.5)Trueunknown i.e. objectthe original object e.g. cftime.datetime(...FalseIterating when
valuesis any kind of date-timeYou can't iterate through a single date-time object!
Fix
The easy part
Switch back to
values.dtype == objectand the test will pass again!The hard part
We should probably have test coverage and correct handling of scalar date-time columns. I'd be happy with this being a separate issue, unless you wanted to go for extra credit!
Originally posted by @trexfeathers in #6948 (comment)
Expected behaviour
- Add test to cover situation where
iris.pandas.as_cubesis passed apandasDataFrame with a scalar time column. - Add handling of scalar (non-iterable) time column to
iris.pandas._get_dimensional_metadata.