Revert cublas_fused_mlp.cpp

[liujuncheng]

No description provided.

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8165/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.3ms (= 12933.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 144.1ms (= 14409.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 144.1ms / 129.3ms)

OneFlow resnet50 time: 78.6ms (= 7860.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.7ms (= 8769.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.12 (= 87.7ms / 78.6ms)

OneFlow resnet50 time: 52.3ms (= 10451.6ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.0ms (= 11604.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.11 (= 58.0ms / 52.3ms)

OneFlow resnet50 time: 43.9ms (= 8778.8ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 46.3ms (= 9257.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.05 (= 46.3ms / 43.9ms)

OneFlow resnet50 time: 38.5ms (= 7706.1ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.1ms (= 7817.6ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.01 (= 39.1ms / 38.5ms)

OneFlow swin dataloader time: 0.258s (= 51.589s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.378s / 200, num_workers=1)
Relative speed: 0.589 (= 0.152s / 0.258s)

OneFlow swin dataloader time: 0.070s (= 14.045s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.448s / 200, num_workers=4)
Relative speed: 0.601 (= 0.042s / 0.070s)

OneFlow swin dataloader time: 0.037s (= 7.493s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.418s / 200, num_workers=8)
Relative speed: 0.590 (= 0.022s / 0.037s)

❌ OneFlow resnet50 time: 145.0ms (= 14499.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 169.3ms (= 16928.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 169.3ms / 145.0ms)

OneFlow resnet50 time: 96.4ms (= 9638.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 108.5ms (= 10845.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 108.5ms / 96.4ms)

OneFlow resnet50 time: 75.4ms (= 15073.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 84.5ms (= 16906.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.12 (= 84.5ms / 75.4ms)

OneFlow resnet50 time: 63.1ms (= 12622.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.9ms (= 14988.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 74.9ms / 63.1ms)

OneFlow resnet50 time: 56.8ms (= 11363.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.6ms (= 13712.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 68.6ms / 56.8ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8165/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.4ms (= 12942.1ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.5ms (= 14348.5ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 143.5ms / 129.4ms)

OneFlow resnet50 time: 79.4ms (= 7939.5ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.3ms (= 8534.9ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.07 (= 85.3ms / 79.4ms)

OneFlow resnet50 time: 52.1ms (= 10418.1ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.0ms (= 11601.6ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.11 (= 58.0ms / 52.1ms)

OneFlow resnet50 time: 43.5ms (= 8697.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 41.2ms (= 8236.2ms / 200, input_shape=[2, 3, 224, 224])
❌ Relative speed: 0.95 (= 41.2ms / 43.5ms)

OneFlow resnet50 time: 36.7ms (= 7331.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 40.6ms (= 8129.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.11 (= 40.6ms / 36.7ms)

OneFlow swin dataloader time: 0.258s (= 51.548s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.499s / 200, num_workers=1)
Relative speed: 0.592 (= 0.152s / 0.258s)

OneFlow swin dataloader time: 0.069s (= 13.797s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.348s / 200, num_workers=4)
Relative speed: 0.605 (= 0.042s / 0.069s)

OneFlow swin dataloader time: 0.039s (= 7.719s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.469s / 200, num_workers=8)
Relative speed: 0.579 (= 0.022s / 0.039s)

❌ OneFlow resnet50 time: 144.9ms (= 14494.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 166.6ms (= 16656.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 166.6ms / 144.9ms)

OneFlow resnet50 time: 96.8ms (= 9678.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.4ms (= 11238.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 112.4ms / 96.8ms)

OneFlow resnet50 time: 76.4ms (= 15289.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 87.6ms (= 17513.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.15 (= 87.6ms / 76.4ms)

OneFlow resnet50 time: 62.1ms (= 12423.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.0ms (= 15001.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 75.0ms / 62.1ms)

OneFlow resnet50 time: 57.1ms (= 11420.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.6ms (= 15710.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.38 (= 78.6ms / 57.1ms)