Nn utils spectral norm new

[ehuaa]

看了oneflow上之前spectral norm一些同学的相关照搬pytorch的提交有以下问题，可以精简
1.增加 spectral_norm 的基本功能, 查看pytorch 实现spectral_norm的源代码中的很多注释，
其实很多 SpectralNormStateDictHook以及SpectralNormLoadStateDictPreHook 因为涉及到和之前出现bug的老版本spectral_norm的线上代码向下兼容所以才需要添加，oneflow中也不需要。
2. Pytorch中很多代码bug相关的修改是针对其特殊架构下的DataParallel模式下的改动，Oneflow中并不需要做相应修改。

pytorch/pytorch@2cd912b
pytorch/pytorch@ac994f2
代码格式化 - √
已经本地编译通过 - √
已本地针对改动测试 - √
添加 enhancement 标签 - √
Draft 转正式 PR 前已请人Review - √

[CLAassistant]

All committers have signed the CLA.

[github-actions]

CI failed when running job: Build cpu. PR label automerge has been removed

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

❌ OneFlow resnet50 time: 141.3ms (= 14130.3ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 144.1ms (= 14413.5ms / 100, input_shape=[16, 3, 224, 224])
❌ Relative speed: 1.02 (= 144.1ms / 141.3ms)

OneFlow resnet50 time: 83.0ms (= 8297.7ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.7ms (= 8768.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.06 (= 87.7ms / 83.0ms)

OneFlow resnet50 time: 50.9ms (= 10180.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 55.8ms (= 11153.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.10 (= 55.8ms / 50.9ms)

OneFlow resnet50 time: 33.9ms (= 6779.6ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 44.2ms (= 8836.4ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.30 (= 44.2ms / 33.9ms)

OneFlow resnet50 time: 25.3ms (= 5059.7ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 37.6ms (= 7516.6ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.49 (= 37.6ms / 25.3ms)

OneFlow swin dataloader time: 0.236s (= 47.117s / 200, num_workers=1)
PyTorch swin dataloader time: 0.148s (= 29.684s / 200, num_workers=1)
Relative speed: 0.630 (= 0.148s / 0.236s)

OneFlow swin dataloader time: 0.065s (= 13.040s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.278s / 200, num_workers=4)
Relative speed: 0.635 (= 0.041s / 0.065s)

OneFlow swin dataloader time: 0.043s (= 8.604s / 200, num_workers=8)
PyTorch swin dataloader time: 0.023s (= 4.520s / 200, num_workers=8)
Relative speed: 0.525 (= 0.023s / 0.043s)

❌ OneFlow resnet50 time: 153.5ms (= 15351.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 165.6ms (= 16557.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.08 (= 165.6ms / 153.5ms)

OneFlow resnet50 time: 93.7ms (= 9369.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 103.5ms (= 10348.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.10 (= 103.5ms / 93.7ms)

OneFlow resnet50 time: 61.6ms (= 12316.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.0ms (= 15590.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.27 (= 78.0ms / 61.6ms)

OneFlow resnet50 time: 43.9ms (= 8771.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.3ms (= 13861.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.58 (= 69.3ms / 43.9ms)

OneFlow resnet50 time: 37.1ms (= 7420.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.3ms (= 13869.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.87 (= 69.3ms / 37.1ms)

[hhhfccz]

您好，可以查看 #9674 完全对齐pytorch仍需要部分算子编写，欢迎您参照该PR进行开发，如果有建议欢迎在 #9674 讨论