Add cu12 release

[jackalcooper]

No description provided.

[github-actions]

View latest API docs preview at: https://oneflow-staging.oss-cn-beijing.aliyuncs.com/docs/Oneflow-Inc/oneflow/pr/10348/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.4ms (= 4340.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 57.2ms (= 5724.1ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.32 (= 57.2ms / 43.4ms)

OneFlow resnet50 time: 26.2ms (= 2619.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.9ms (= 3785.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.45 (= 37.9ms / 26.2ms)

OneFlow resnet50 time: 19.2ms (= 3844.5ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 35.7ms (= 7148.2ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.86 (= 35.7ms / 19.2ms)

OneFlow resnet50 time: 18.3ms (= 3667.5ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 31.3ms (= 6259.7ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.71 (= 31.3ms / 18.3ms)

OneFlow resnet50 time: 19.0ms (= 3791.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 28.1ms (= 5611.8ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.48 (= 28.1ms / 19.0ms)

OneFlow swin dataloader time: 0.200s (= 40.020s / 200, num_workers=1)
PyTorch swin dataloader time: 0.130s (= 25.932s / 200, num_workers=1)
Relative speed: 0.648 (= 0.130s / 0.200s)

OneFlow swin dataloader time: 0.055s (= 10.928s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.610s / 200, num_workers=4)
Relative speed: 0.605 (= 0.033s / 0.055s)

OneFlow swin dataloader time: 0.031s (= 6.174s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.411s / 200, num_workers=8)
Relative speed: 0.552 (= 0.017s / 0.031s)

❌ OneFlow resnet50 time: 47.8ms (= 4781.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 64.5ms (= 6449.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 64.5ms / 47.8ms)

OneFlow resnet50 time: 31.5ms (= 3145.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 43.7ms (= 4365.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.39 (= 43.7ms / 31.5ms)

OneFlow resnet50 time: 23.9ms (= 4781.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 43.2ms (= 8645.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.81 (= 43.2ms / 23.9ms)

OneFlow resnet50 time: 23.4ms (= 4677.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 38.2ms (= 7631.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.63 (= 38.2ms / 23.4ms)

OneFlow resnet50 time: 21.6ms (= 4310.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 34.7ms (= 6938.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.61 (= 34.7ms / 21.6ms)