Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add more vision model compatibility tests #7942

Merged
merged 57 commits into from
Apr 15, 2022
Merged

add more vision model compatibility tests #7942

merged 57 commits into from
Apr 15, 2022

Conversation

Ldpe2G
Copy link
Contributor

@Ldpe2G Ldpe2G commented Apr 1, 2022 •
edited
Loading

添加的模型：

修复不兼容的接口

  • conv2d module padding 参数不支持 string
  • functional.layer_norm 参数列表没对齐
  • meshgrid 不支持 list[tensor] 输入
  • 添加 tensor.reshape_as 接口

@Ldpe2G Ldpe2G requested a review from rentainhe April 1, 2022 01:23
@Ldpe2G Ldpe2G requested a review from oneflow-ci-bot April 1, 2022 01:30
@Ldpe2G Ldpe2G requested review from oneflow-ci-bot and removed request for jackalcooper, daquexian and oneflow-ci-bot April 1, 2022 10:14
@github-actions
Copy link
Contributor

github-actions bot commented Apr 1, 2022

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

@Ldpe2G Ldpe2G requested review from oneflow-ci-bot and removed request for oneflow-ci-bot April 1, 2022 10:20
@Ldpe2G Ldpe2G requested review from oneflow-ci-bot and removed request for oneflow-ci-bot April 1, 2022 15:03
@github-actions
Copy link
Contributor

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7942/

@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.7ms (= 12874.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.8ms (= 14082.5ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.09 (= 140.8ms / 128.7ms)

OneFlow resnet50 time: 77.3ms (= 7725.0ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.4ms (= 8344.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.08 (= 83.4ms / 77.3ms)

OneFlow resnet50 time: 53.0ms (= 10607.1ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 57.6ms (= 11520.0ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.09 (= 57.6ms / 53.0ms)

OneFlow resnet50 time: 43.6ms (= 8726.0ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 43.2ms (= 8643.8ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 0.99 (= 43.2ms / 43.6ms)

OneFlow resnet50 time: 39.2ms (= 7836.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 43.3ms (= 8653.1ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.10 (= 43.3ms / 39.2ms)

OneFlow swin dataloader time: 0.246s (= 49.239s / 200, num_workers=1)
PyTorch swin dataloader time: 0.249s (= 49.856s / 200, num_workers=1)
✔️ Relative speed: 1.013 (= 0.249s / 0.246s)

OneFlow swin dataloader time: 0.065s (= 12.954s / 200, num_workers=4)
PyTorch swin dataloader time: 0.068s (= 13.501s / 200, num_workers=4)
✔️ Relative speed: 1.042 (= 0.068s / 0.065s)

OneFlow swin dataloader time: 0.036s (= 7.263s / 200, num_workers=8)
PyTorch swin dataloader time: 0.036s (= 7.213s / 200, num_workers=8)
✔️ Relative speed: 0.993 (= 0.036s / 0.036s)

✔️ OneFlow resnet50 time: 135.2ms (= 13519.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 156.0ms (= 15598.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 156.0ms / 135.2ms)

OneFlow resnet50 time: 86.9ms (= 8690.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 98.6ms (= 9855.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 98.6ms / 86.9ms)

OneFlow resnet50 time: 62.6ms (= 12520.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.2ms (= 15049.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 75.2ms / 62.6ms)

OneFlow resnet50 time: 57.0ms (= 11399.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.3ms (= 13067.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.15 (= 65.3ms / 57.0ms)

OneFlow resnet50 time: 51.2ms (= 10244.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 61.1ms (= 12213.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 61.1ms / 51.2ms)

@github-actions
Copy link
Contributor

CI failed when running job: cuda-speed-test. PR label automerge has been removed

@github-actions
Copy link
Contributor

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7942/

@github-actions
Copy link
Contributor

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7942/

@github-actions
Copy link
Contributor

Speed stats:

@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.8ms (= 12884.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 137.7ms (= 13771.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.07 (= 137.7ms / 128.8ms)

OneFlow resnet50 time: 82.7ms (= 8274.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.7ms (= 8370.6ms / 100, input_shape=[8, 3, 224, 224])
❌ Relative speed: 1.01 (= 83.7ms / 82.7ms)

OneFlow resnet50 time: 51.3ms (= 10269.5ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 53.8ms (= 10757.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.05 (= 53.8ms / 51.3ms)

OneFlow resnet50 time: 41.5ms (= 8298.9ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 48.3ms (= 9668.8ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.17 (= 48.3ms / 41.5ms)

OneFlow resnet50 time: 39.7ms (= 7948.6ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 38.3ms (= 7657.6ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.96 (= 38.3ms / 39.7ms)

OneFlow swin dataloader time: 0.249s (= 49.802s / 200, num_workers=1)
PyTorch swin dataloader time: 0.252s (= 50.392s / 200, num_workers=1)
✔️ Relative speed: 1.012 (= 0.252s / 0.249s)

OneFlow swin dataloader time: 0.069s (= 13.727s / 200, num_workers=4)
PyTorch swin dataloader time: 0.069s (= 13.760s / 200, num_workers=4)
✔️ Relative speed: 1.002 (= 0.069s / 0.069s)

OneFlow swin dataloader time: 0.037s (= 7.493s / 200, num_workers=8)
PyTorch swin dataloader time: 0.037s (= 7.360s / 200, num_workers=8)
✔️ Relative speed: 0.982 (= 0.037s / 0.037s)

✔️ OneFlow resnet50 time: 135.3ms (= 13531.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 156.3ms (= 15629.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 156.3ms / 135.3ms)

OneFlow resnet50 time: 86.9ms (= 8693.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 106.0ms (= 10604.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 106.0ms / 86.9ms)

OneFlow resnet50 time: 62.2ms (= 12443.2ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.0ms (= 15192.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 76.0ms / 62.2ms)

OneFlow resnet50 time: 53.1ms (= 10611.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 66.8ms (= 13350.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 66.8ms / 53.1ms)

OneFlow resnet50 time: 50.3ms (= 10066.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 61.2ms (= 12240.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 61.2ms / 50.3ms)

@github-actions
Copy link
Contributor

CI failed when running job: cuda-speed-test. PR label automerge has been removed

@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.6ms (= 12860.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 144.9ms (= 14485.6ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.13 (= 144.9ms / 128.6ms)

OneFlow resnet50 time: 80.7ms (= 8074.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 86.4ms (= 8635.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.07 (= 86.4ms / 80.7ms)

OneFlow resnet50 time: 54.8ms (= 10969.5ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.1ms (= 11623.0ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.06 (= 58.1ms / 54.8ms)

OneFlow resnet50 time: 44.9ms (= 8987.5ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 50.2ms (= 10047.1ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.12 (= 50.2ms / 44.9ms)

OneFlow resnet50 time: 40.2ms (= 8048.3ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.4ms (= 7882.0ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.98 (= 39.4ms / 40.2ms)

OneFlow swin dataloader time: 0.254s (= 50.732s / 200, num_workers=1)
PyTorch swin dataloader time: 0.251s (= 50.137s / 200, num_workers=1)
✔️ Relative speed: 0.988 (= 0.251s / 0.254s)

OneFlow swin dataloader time: 0.069s (= 13.701s / 200, num_workers=4)
PyTorch swin dataloader time: 0.074s (= 14.716s / 200, num_workers=4)
✔️ Relative speed: 1.074 (= 0.074s / 0.069s)

OneFlow swin dataloader time: 0.037s (= 7.428s / 200, num_workers=8)
PyTorch swin dataloader time: 0.037s (= 7.305s / 200, num_workers=8)
✔️ Relative speed: 0.983 (= 0.037s / 0.037s)

✔️ OneFlow resnet50 time: 136.1ms (= 13613.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 156.5ms (= 15646.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 156.5ms / 136.1ms)

OneFlow resnet50 time: 89.4ms (= 8941.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 102.9ms (= 10288.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 102.9ms / 89.4ms)

OneFlow resnet50 time: 62.4ms (= 12475.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.8ms (= 15354.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.23 (= 76.8ms / 62.4ms)

OneFlow resnet50 time: 53.2ms (= 10636.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 67.4ms (= 13485.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.27 (= 67.4ms / 53.2ms)

OneFlow resnet50 time: 48.0ms (= 9609.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 61.6ms (= 12328.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.28 (= 61.6ms / 48.0ms)

@github-actions
Copy link
Contributor

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7942/

@jackalcooper jackalcooper merged commit 39cd242 into master Apr 15, 2022
@jackalcooper jackalcooper deleted the dev_add_vision_model_compatiblity_tests branch April 15, 2022 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@doombeaker doombeaker doombeaker left review comments

@BBuf BBuf BBuf approved these changes

@rentainhe rentainhe rentainhe approved these changes

@oneflow-ci-bot oneflow-ci-bot Awaiting requested review from oneflow-ci-bot

Assignees
No one assigned
Labels
api automerge ci eager enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@Ldpe2G @doombeaker @BBuf @rentainhe @jackalcooper @daquexian @oneflow-ci-bot