Making PLKSR stable for real-world SISR #4
Description
Hi. First of all, thanks to everyone who participated on this research. Very thorough analysis on the paper.
As reported by others in issue 3, PLKSR seems to be unstable for real-world SISR. GAN training is notoriously unstable, and causes issues even at lower learning rate.
So in an attempt to make it more stable, I have released a simple modification to PLKSR, named RealPLKSR:
- Normalization was missing, as pointed by @dslisleedh. From my understanding, layer norm was avoided because of the impact on inference latency. I have tested multiple methods, including Instance norm, Layer norm, Batch norm, Group norm and RMSNorm. Because we usually train at lower batch sizes (<16), out of those tested, Group Normalization performed best on my experiments. The impact on inference latency was minimal (~5% max). The number of groups was also tested. Increasing it leads to better regularization, but impacts convergence speed. The value
4offered a good balance on all tests. - Replacing GELU with Mish on channel mixer. Mish showed better, more stable, convergence compared to GELU.
- Added
nn.Dropout2dto the last conv, as proposed in "Reflash Dropout in Image Super-Resolution". Although not ideal, dropout is a simple method to increase generalization on real-world SISR.
Pretrained models:
| scale | download |
|---|---|
| 4x GAN | GDrive |
| 4x | GDrive |
| 2x | GDrive |
Training can be done on neosr using the following configurations: paired dataset or through realesrgan degradation pipeline.
Credits were acknowledged inside the code and released under the same license as PLKSR (MIT). I hope this makes PLKSR more used under real-world degradations. It's a really impressive network. Thanks again for your research 👍