Expanding Window Cross-Validation for Time Series?

[Cajeux1999]

Does TPOT supports Cross-Validation for Time Series using an Expanding Window, as shown below?

I am thinking on how that could be done, but i still haven't found a solution yet.

[perib]

It's not built in, but TPOT now supports custom objective functions and cv splitters.

Two options:

1. (simplest) you pass in an sklearn cv splitter that does this strategy (sklearn.model_selection.BaseCrossValidator) to the 'cv' parameter. I think 'cv=sklearn.model_selection.TimeSeriesSplit(n_splits=10)' should be what you are looking for.

You can define a custom function that takes in a model, and returns the expanded window cross validation score. Then pass that function into 'other_objective_functions'. Make sure to also set 'other_objective_functions_weights'

See tutorial 1 here for an example:

https://github.com/EpistasisLab/tpot/blob/main/Tutorial/1_Using_TPOT.ipynb

[Cajeux1999]

Hi Pedro,

I have used TPOT for an article submitted to a local conference, applied to time-series forecasting. Results were very good.

I can upload an example notebook on TPOT applied to time series, applied in a expanding window setup.

[perib]

This is so cool! I'm glad this version of TPOT2 is enabling novel work like yours.

I think this could potentially be a good notebook for the Tutorials folder? What do others think?