consistent matmul test case

[liufengwei0103]

No description provided.

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

CI failed when running job: cuda-module-distributed. PR label automerge has been removed

[github-actions]

CI failed when running job: cuda-module-distributed. PR label automerge has been removed

[jackalcooper]

这个PR多机死锁了

[liufengwei0103]

这个PR多机死锁了

这个时间本地测试也是超长时间，我减小下规模吧。

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.6ms (= 12855.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 137.2ms (= 13718.1ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.07 (= 137.2ms / 128.6ms)

✔️ OneFlow resnet50 time: 78.7ms (= 7867.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.1ms (= 8706.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.11 (= 87.1ms / 78.7ms)

OneFlow resnet50 time: 53.3ms (= 10653.8ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 60.8ms (= 12160.0ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.14 (= 60.8ms / 53.3ms)

OneFlow resnet50 time: 44.4ms (= 8878.5ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.7ms (= 9944.7ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.12 (= 49.7ms / 44.4ms)

OneFlow resnet50 time: 39.7ms (= 7942.2ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.1ms (= 7816.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.98 (= 39.1ms / 39.7ms)

✔️ OneFlow resnet50 time: 142.6ms (= 14260.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.6ms (= 16063.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 160.6ms / 142.6ms)

OneFlow resnet50 time: 88.1ms (= 8805.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 102.0ms (= 10204.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 102.0ms / 88.1ms)

OneFlow resnet50 time: 61.6ms (= 12319.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.9ms (= 15180.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.23 (= 75.9ms / 61.6ms)

OneFlow resnet50 time: 52.2ms (= 10444.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.4ms (= 13083.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 65.4ms / 52.2ms)

OneFlow resnet50 time: 47.3ms (= 9464.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 59.3ms (= 11857.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 59.3ms / 47.3ms)