[Primitive] Unit test

[liujuncheng]

No description provided.

[github-actions]

CI failed when running job: Build cu102_xla. PR label automerge has been removed

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8099/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.4ms (= 12935.9ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 142.9ms (= 14292.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 142.9ms / 129.4ms)

OneFlow resnet50 time: 83.7ms (= 8369.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.9ms (= 8590.6ms / 100, input_shape=[8, 3, 224, 224])
❌ Relative speed: 1.03 (= 85.9ms / 83.7ms)

OneFlow resnet50 time: 54.1ms (= 10829.6ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.7ms (= 11732.5ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.08 (= 58.7ms / 54.1ms)

OneFlow resnet50 time: 43.2ms (= 8632.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 47.9ms (= 9587.1ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.11 (= 47.9ms / 43.2ms)

OneFlow resnet50 time: 36.6ms (= 7324.0ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 40.6ms (= 8116.2ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.11 (= 40.6ms / 36.6ms)

OneFlow swin dataloader time: 0.259s (= 51.874s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.380s / 200, num_workers=1)
Relative speed: 0.586 (= 0.152s / 0.259s)

OneFlow swin dataloader time: 0.068s (= 13.532s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.377s / 200, num_workers=4)
Relative speed: 0.619 (= 0.042s / 0.068s)

OneFlow swin dataloader time: 0.037s (= 7.486s / 200, num_workers=8)
PyTorch swin dataloader time: 0.023s (= 4.550s / 200, num_workers=8)
Relative speed: 0.608 (= 0.023s / 0.037s)

❌ OneFlow resnet50 time: 145.1ms (= 14508.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 167.6ms (= 16761.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 167.6ms / 145.1ms)

OneFlow resnet50 time: 96.0ms (= 9598.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.9ms (= 11291.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 112.9ms / 96.0ms)

OneFlow resnet50 time: 78.5ms (= 15693.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.9ms (= 17778.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.13 (= 88.9ms / 78.5ms)

OneFlow resnet50 time: 63.4ms (= 12670.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.0ms (= 15196.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 76.0ms / 63.4ms)

OneFlow resnet50 time: 55.8ms (= 11161.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.0ms (= 13792.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.24 (= 69.0ms / 55.8ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8099/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.3ms (= 12925.1ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.8ms (= 14183.5ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 141.8ms / 129.3ms)

OneFlow resnet50 time: 78.8ms (= 7880.2ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.8ms (= 8381.2ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.06 (= 83.8ms / 78.8ms)

OneFlow resnet50 time: 53.2ms (= 10632.0ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 64.1ms (= 12826.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.21 (= 64.1ms / 53.2ms)

OneFlow resnet50 time: 40.6ms (= 8123.7ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 48.6ms (= 9722.1ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.20 (= 48.6ms / 40.6ms)

OneFlow resnet50 time: 39.4ms (= 7881.2ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 36.9ms (= 7388.8ms / 200, input_shape=[1, 3, 224, 224])
❌ Relative speed: 0.94 (= 36.9ms / 39.4ms)

OneFlow swin dataloader time: 0.257s (= 51.423s / 200, num_workers=1)
PyTorch swin dataloader time: 0.154s (= 30.890s / 200, num_workers=1)
Relative speed: 0.601 (= 0.154s / 0.257s)

OneFlow swin dataloader time: 0.110s (= 21.952s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.162s / 200, num_workers=4)
Relative speed: 0.372 (= 0.041s / 0.110s)

OneFlow swin dataloader time: 0.037s (= 7.334s / 200, num_workers=8)
PyTorch swin dataloader time: 0.023s (= 4.699s / 200, num_workers=8)
Relative speed: 0.641 (= 0.023s / 0.037s)

❌ OneFlow resnet50 time: 144.8ms (= 14480.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 168.0ms (= 16796.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 168.0ms / 144.8ms)

OneFlow resnet50 time: 100.1ms (= 10009.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.6ms (= 11259.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.12 (= 112.6ms / 100.1ms)

OneFlow resnet50 time: 74.5ms (= 14892.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.3ms (= 17656.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 88.3ms / 74.5ms)

OneFlow resnet50 time: 64.8ms (= 12953.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 85.4ms (= 17086.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.32 (= 85.4ms / 64.8ms)

OneFlow resnet50 time: 55.2ms (= 11049.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.0ms (= 13807.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 69.0ms / 55.2ms)