Critical Difference Diagrams

This repository contains the necessary code to generate a critical difference diagram based on the Wilcoxon-Holm method to detect pairwise significance.

Result

By running the python3 main.py you will generate a critical difference diagram with Wilcoxon-Holm post-hoc analysis for the data present in the example.csv file. First the Friedman test is performed to reject the null hypothesis, we then proceed with a post-hoc analysis based on the Wilcoxon-Holm method. We can clearly see how on average clf3 and clf5 were the best algorithms over the 15 datasets. A thick horizontal line groups a set of classifiers that are not significantly different.

Relevant projects

Deep learning for time series classification: a review

In this paper we used the critical difference diagram to compare the recent deep learning models for time series classification where we evaluated 9 different architectures on 85 different datasets from the UCR/UEA archive.

Check out the code!

Deep Neural Network Ensembles for Time Series Classification

In this paper we used the critical difference diagram to show how ensembling hybrid architectures will allow deep learning models to reach even better accuracy when evaluated on 85 different datasets from the UCR/UEA archive.

Check out the code!

Requirements

To run this code you will need the following python packages:

• numpy
• pandas
• matplotlib
• scipy
• networkx

Reference

If you re-use this work, please cite:

@article{IsmailFawaz2018deep,
  Title                    = {Deep learning for time series classification: a review},
  Author                   = {Ismail Fawaz, Hassan and Forestier, Germain and Weber, Jonathan and Idoumghar, Lhassane and Muller, Pierre-Alain},
  journal                  = {Data Mining and Knowledge Discovery},
  Year                     = {2019},
  volume                   = {33},
  number                   = {4},
  pages                    = {917--963},
}