1x1 convolutional layers in shortcuts in ResNet-18 are pruned completely during singleshot SynFlow

[xhchrn]

In single-shot experiments using SynFlow as the pruner, it is very likely that the 1x1 convolutional layers are completely pruned out. A command to reproduce this observation is:

python main.py --dataset tiny-imagenet --model resnet18 --model-class tinyimagenet --compression 1.0 --experiment singleshot --pruner synflow --prune-epochs 100

I observe similar things on CIFAR-10 using ResNet-18 (which is not implemented in this repo). The shortcut convolutional layers in ResNet-20 on CIFAR-10 will not be pruned completely, but very close, leaving only a few non-zero elements.