ENH Add Wide ResNet

[zaccharieramzi]

Wide ResNet seem to be a very popular variant of ResNets, achieving sota in many benchmarks such as CIFAR-100, and being used in recent research papers in big institutions (like DeepMind, Google), sometimes with variants on init, regularization and data augmentation.

[zaccharieramzi]

With the torch version I recover the SOTA results indicated on papers with code, which are:

• CIFAR: 96.15
• CIFAR-100: 81.35

[zaccharieramzi]

Currently needs this version of tf2cv : https://github.com/zaccharieramzi/imgclsmob/tree/config-wrn, for the TF WRN.

This is because the serialization that we need (in order to apply coupled weight decay), does not work with the pypi version

Also, it does not work for tf<=2.8

EDIT

Now my custom version of tf2cv is not needed anymore for the TF WRN to work in this benchmark.

[zaccharieramzi]

For now the TF version does not work: it's not learning despite using the exact same parameters than PyTorch on CIFAR-10.

[zaccharieramzi]

The problem is the following:
Basically, when applying the coupled weight decay in TensorFlow, we need to manually set it on all the layers of the model (when applicable).
The problem with the wide resnet implemented in imgclsmob, is that it relies on specific layers that are "difficult" to inspect in the sense that it would require a custom way of inspecting them.

EDIT

Actually, we can work something out, and it would be helpful to handle models that are not exactly "flat". I think this makes the application of coupled weight decay more robust.

[zaccharieramzi]

TF version of WRN now working (96.08 on CIFAR-10) -> this is ready to merge.
I think we can have the model consistency as an other issue in the pipe.