[BUG] PLAID Dataset is being loaded as unequal always

[hadifawaz1999]

Describe the bug

When loading the PLAID dataset (originally unequal length) with load_classification it is always loading the list version never the equal length version

Steps/Code to reproduce the bug

x, y = load_classification(name="PLAID")
print(x.shape)

Expected results

shape output

Actual results

Attribute error, list does not have attribute shape

Versions

No response

[lucifer4073]

Hi @hadifawaz1999, I would like to work on this issue. Can you please assign it to me?

[MatthewMiddlehurst]

@lucifer4073 Please have a look through the contributing guide. This issue involved loading external data so theres a chance you wont be able to fix this.

[lucifer4073]

Hi @MatthewMiddlehurst, thank you for your response. I attempted to debug the issue and found that x has an inhomogeneous shape, which prevents it from being converted into a NumPy array. This seems to stem from the external data being loaded.

Type of x -> <class 'list'>
Type of y -> <class 'numpy.ndarray'>
Type of x[0] -> <class 'numpy.ndarray'>
Type of x[0][0] -><class 'numpy.ndarray'>

I’ve attached this for reference in case someone else wants to review or provide further insights.

Attempt: x=np.asarray(x)

Result: ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (1074, 1) + inhomogeneous part.

[MatthewMiddlehurst]

The function has a load_equal_length parameter which should handle this, but I don't think its an issue with the function itself.

[hadifawaz1999]

@TonyBagnall what do you think

[saadaltohamy]

Hi, I found that the Plaid dataset doesn't have _disc, _eq, or _nmv versions. So when calling _load_saved_dataset(), it returns a list of 2D arrays due to variable-length multivariate series.

Shape of the nested list:
(1074,)
Shapes of the first 5 elements in X:
(1, 500)
(1, 500)
(1, 500)
(1, 200)
(1, 544)

Trying to convert this list to a 3D NumPy array with _from_np_list_to_numpy3d() raises:
ValueError: Cannot convert unequal length to numpy3D.

I think we could handle this by padding sequences when this error is raised. If that sounds good, I’m happy to open a PR to add a padding utility. @hadifawaz1999 @MatthewMiddlehurst @TonyBagnall

[MatthewMiddlehurst]

No. Automatically padding data like this is not something we want to do IMO.

[hadifawaz1999]

yeah i agree with @MatthewMiddlehurst it is actually an issue from the timeseriesclassification.com website @TonyBagnall probably the equal length version is not uploaded. Plus if am not mistaken (correct me @MatthewMiddlehurst @TonyBagnall ) the equal length version is not padding zero, it is padding the mean value per series followed by adding small gaussian noise on the padding places, this was Eamonn's solution on the ucr website, so i assumed you did the same.

[MatthewMiddlehurst]

Generally the correct way to transform a dataset is highly dependent on the data itself. Unless it is an archive dataset (which is the case here, but as Ali said its a website issue), or part of an algorithm, we should not be doing any automatic preprocessing.

[MatthewMiddlehurst]

I dont quite remember what we did for the dataset, but yeah its usually zero or mean with low level noise or truncation

[saadaltohamy]

Is it allowed for the function to return datasets with variable-length time series? e.g., list of ndarrays that have variable length
@MatthewMiddlehurst @hadifawaz1999

[saadaltohamy]

As I see in docstring, I think the answer is yes.

So yeah, we don't have a bug here, at least in the function.