Dev Constant Pad Primitive [Primitve]

[MARD1NO]

使用Pad Primitve，优化性能实现

移除过时的PadGrad constant模式，直接将pad取反，利用Pad来做反向

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8152/

[github-actions]

Speed stats:

[github-actions]

CI failed when running job: cuda-benchmark. PR label automerge has been removed

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8152/

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.4ms (= 12843.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 144.2ms (= 14420.7ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.12 (= 144.2ms / 128.4ms)

OneFlow resnet50 time: 75.3ms (= 7534.0ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.8ms (= 8376.6ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.11 (= 83.8ms / 75.3ms)

OneFlow resnet50 time: 51.8ms (= 10362.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 62.9ms (= 12575.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.21 (= 62.9ms / 51.8ms)

OneFlow resnet50 time: 40.0ms (= 8008.7ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 39.5ms (= 7908.1ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 0.99 (= 39.5ms / 40.0ms)

OneFlow resnet50 time: 34.5ms (= 6904.0ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.4ms (= 7876.9ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.14 (= 39.4ms / 34.5ms)

OneFlow swin dataloader time: 0.245s (= 49.057s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.427s / 200, num_workers=1)
Relative speed: 0.620 (= 0.152s / 0.245s)

OneFlow swin dataloader time: 0.065s (= 13.099s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.518s / 200, num_workers=4)
Relative speed: 0.650 (= 0.043s / 0.065s)

OneFlow swin dataloader time: 0.036s (= 7.170s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.495s / 200, num_workers=8)
Relative speed: 0.627 (= 0.022s / 0.036s)

❌ OneFlow resnet50 time: 136.5ms (= 13646.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.9ms (= 16093.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 160.9ms / 136.5ms)

OneFlow resnet50 time: 86.4ms (= 8643.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 99.9ms (= 9987.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 99.9ms / 86.4ms)

OneFlow resnet50 time: 62.1ms (= 12425.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 83.5ms (= 16693.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.34 (= 83.5ms / 62.1ms)

OneFlow resnet50 time: 51.8ms (= 10363.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 67.8ms (= 13552.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.31 (= 67.8ms / 51.8ms)

OneFlow resnet50 time: 47.4ms (= 9475.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 67.3ms (= 13466.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.42 (= 67.3ms / 47.4ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8152/