Improve VectorAt

[daquexian]

VectorAt 支持传入非常量引用：

JUST(VectorAt(vec, 1)) = 5;

[github-actions]

CI failed when running job: cuda-module. PR label automerge has been removed

[github-actions]

CI failed when running job: cuda-module. PR label automerge has been removed

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8013/

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 129.0ms (= 12904.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.2ms (= 14324.5ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 143.2ms / 129.0ms)

OneFlow resnet50 time: 83.1ms (= 8305.7ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.3ms (= 8428.5ms / 100, input_shape=[8, 3, 224, 224])
❌ Relative speed: 1.01 (= 84.3ms / 83.1ms)

OneFlow resnet50 time: 52.8ms (= 10557.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 56.9ms (= 11383.7ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.08 (= 56.9ms / 52.8ms)

OneFlow resnet50 time: 42.7ms (= 8541.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 47.0ms (= 9397.7ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.10 (= 47.0ms / 42.7ms)

OneFlow resnet50 time: 39.7ms (= 7932.9ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 40.5ms (= 8105.7ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.02 (= 40.5ms / 39.7ms)

OneFlow swin dataloader time: 0.244s (= 48.773s / 200, num_workers=1)
PyTorch swin dataloader time: 0.150s (= 29.959s / 200, num_workers=1)
Relative speed: 0.614 (= 0.150s / 0.244s)

OneFlow swin dataloader time: 0.070s (= 13.910s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.215s / 200, num_workers=4)
Relative speed: 0.591 (= 0.041s / 0.070s)

OneFlow swin dataloader time: 0.037s (= 7.366s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.414s / 200, num_workers=8)
Relative speed: 0.599 (= 0.022s / 0.037s)

✔️ OneFlow resnet50 time: 135.8ms (= 13575.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.7ms (= 16166.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 161.7ms / 135.8ms)

OneFlow resnet50 time: 90.7ms (= 9071.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 111.8ms (= 11183.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.23 (= 111.8ms / 90.7ms)

OneFlow resnet50 time: 64.4ms (= 12885.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 86.7ms (= 17332.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 86.7ms / 64.4ms)

OneFlow resnet50 time: 52.5ms (= 10490.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.9ms (= 13771.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.31 (= 68.9ms / 52.5ms)

OneFlow resnet50 time: 47.1ms (= 9417.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.6ms (= 13715.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.46 (= 68.6ms / 47.1ms)

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.5ms (= 12946.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 142.4ms (= 14239.7ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 142.4ms / 129.5ms)

OneFlow resnet50 time: 83.2ms (= 8320.8ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.3ms (= 8430.1ms / 100, input_shape=[8, 3, 224, 224])
❌ Relative speed: 1.01 (= 84.3ms / 83.2ms)

OneFlow resnet50 time: 52.7ms (= 10545.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 63.4ms (= 12684.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.20 (= 63.4ms / 52.7ms)

OneFlow resnet50 time: 41.8ms (= 8355.3ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 44.2ms (= 8833.0ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.06 (= 44.2ms / 41.8ms)

OneFlow resnet50 time: 39.3ms (= 7859.3ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 42.9ms (= 8574.1ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.09 (= 42.9ms / 39.3ms)

OneFlow swin dataloader time: 0.418s (= 83.595s / 200, num_workers=1)
PyTorch swin dataloader time: 0.153s (= 30.606s / 200, num_workers=1)
Relative speed: 0.366 (= 0.153s / 0.418s)

OneFlow swin dataloader time: 0.067s (= 13.319s / 200, num_workers=4)
PyTorch swin dataloader time: 0.044s (= 8.713s / 200, num_workers=4)
Relative speed: 0.654 (= 0.044s / 0.067s)

OneFlow swin dataloader time: 0.037s (= 7.488s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.306s / 200, num_workers=8)
Relative speed: 0.575 (= 0.022s / 0.037s)

❌ OneFlow resnet50 time: 144.8ms (= 14483.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 168.0ms (= 16800.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 168.0ms / 144.8ms)

OneFlow resnet50 time: 97.0ms (= 9702.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 111.5ms (= 11149.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 111.5ms / 97.0ms)

OneFlow resnet50 time: 77.0ms (= 15400.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 98.9ms (= 19779.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.28 (= 98.9ms / 77.0ms)

OneFlow resnet50 time: 65.0ms (= 13000.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 83.7ms (= 16732.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.29 (= 83.7ms / 65.0ms)

OneFlow resnet50 time: 56.7ms (= 11347.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.3ms (= 13852.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 69.3ms / 56.7ms)