Echo State Networks (ESNs) as a prime example of Reservoir Computing (RC) models are known for their fast and precise one-shot learning of time series. But they often need good hyper-parameter tuning for the best performance. For this good validation is key, but usually a single validation split is used.
This repository contains the code used for the experiments in [1].
Some details of implementation and computational savings depend on what type of task we are learning. Let us distinguish three types of temporal machine learning tasks:
-
Generative tasks, where the computed output y(n) comes back as (part of) the input u(n + k). This is often pattern generation or multi-step timeseries prediction in a generative mode.
-
Output tasks, where the computed output time series y(n) does not comeback as part of input. This is often detection or recognition in time series, or deducing a signal from other contemporary signals.
-
Classification tasks, of separate (pre-cut) finite sequences, where a class y is assigned to each sequence u(n).
For the latter type of tasks we usually store only an averaged or a fixed number of states x(n) for every sequence in the state matrix X. It is similar to a non-temporal classification task.
For each of the learning task we apply our approach to verify its effectiveness. The code for each task is similar, yet divided into separate files to make application to each task clear. As such, each script can be easily alterted to support other datasets with minimal change.
Exact experiments reported in the paper can be replicated by running:
./run_generative_experiments.sh
This will start a job with 4 threads and search for optimal hyper-parameters with 5 different initializations.
After the search ends, all of the results will be saved in precision_experiments/generative_mode/results folder.
Results can be viewed by running show_results.py.
Alternatively, individual experiments, for example, can be run by executing:
python precision_experiments/generative_mode/generative_exp.py --dataset gasoline --valid_method standard --final_model retrain
Full range of options can be seen by adding --help argument.
Same structure was followed as in Generative Mode experiment, see how individual experiments were executed in run_output_exps.sh.
Same structure was followed as in Generative Mode experiment, see how individual experiments were executed in run_classification_exps.sh.
A notebook (ESN_speed_experiment.ipynb) is available to replicate the speed experiment. Alternatively, experiment can be run locally by clicking the Open in Colab button.
[1] Lukoševičius, Mantas, and Arnas Uselis. "Efficient implementations of echo state network cross-validation". 2020. [arxiv]
Please contact Arnas Uselis (auselis gmx com) if any issues regarding the code arise.