Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support constant folding in cpp api #8124

Merged
merged 55 commits into from
May 19, 2022
Merged

Support constant folding in cpp api #8124

merged 55 commits into from
May 19, 2022

Conversation

mosout
Copy link
Contributor

@mosout mosout commented May 2, 2022

No description provided.

@mosout mosout requested a review from oneflow-ci-bot May 3, 2022 03:05
@github-actions
Copy link
Contributor

github-actions bot commented May 3, 2022

CI failed when running job: cpu-misc. PR label automerge has been removed

@github-actions github-actions bot removed the automerge label May 3, 2022
@github-actions
Copy link
Contributor

github-actions bot commented May 3, 2022

Speed stats:

@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 3, 2022 10:03
@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 3, 2022 10:04
mosout and others added 4 commits May 7, 2022 17:48
@mosout
Merge branch 'master' of github.com:Oneflow-Inc/oneflow into fix_cpp_…
54a9056 
…api_mlir
@mosout
Merge branch 'fix_cpp_api_mlir' of github.com:Oneflow-Inc/oneflow int…
08d5e56 
…o fix_cpp_api_mlir
@mosout
refine
27067f7
@BBuf @oneflow-ci-bot
Enable nhwc and constant folding in ci (#8127)
34de714 
* fix eager free tensor bug when in job

* enable nhwc and constant folding in ci

* Update test.yml

* fix batchnorm1d bug

* reert

* auto nhwc pass only exist in cuda mode

* revert

* auto format by CI

* fix bug

* fix comment

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 7, 2022 10:26
@mosout mosout enabled auto-merge (squash) May 7, 2022 10:27
@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 7, 2022 12:19
@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.8ms (= 12983.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 142.1ms (= 14208.1ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.09 (= 142.1ms / 129.8ms)

OneFlow resnet50 time: 77.7ms (= 7771.5ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.6ms (= 8458.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.09 (= 84.6ms / 77.7ms)

OneFlow resnet50 time: 53.1ms (= 10629.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.5ms (= 11909.5ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.12 (= 59.5ms / 53.1ms)

OneFlow resnet50 time: 40.7ms (= 8137.6ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 40.9ms (= 8171.4ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.00 (= 40.9ms / 40.7ms)

OneFlow resnet50 time: 35.7ms (= 7146.9ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 37.0ms (= 7404.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.04 (= 37.0ms / 35.7ms)

OneFlow swin dataloader time: 0.245s (= 48.910s / 200, num_workers=1)
PyTorch swin dataloader time: 0.150s (= 29.937s / 200, num_workers=1)
Relative speed: 0.612 (= 0.150s / 0.245s)

OneFlow swin dataloader time: 0.066s (= 13.123s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.503s / 200, num_workers=4)
Relative speed: 0.648 (= 0.043s / 0.066s)

OneFlow swin dataloader time: 0.035s (= 7.043s / 200, num_workers=8)
PyTorch swin dataloader time: 0.023s (= 4.683s / 200, num_workers=8)
Relative speed: 0.665 (= 0.023s / 0.035s)

❌ OneFlow resnet50 time: 145.5ms (= 14547.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 169.1ms (= 16908.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 169.1ms / 145.5ms)

OneFlow resnet50 time: 97.7ms (= 9765.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.2ms (= 11220.5ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 112.2ms / 97.7ms)

OneFlow resnet50 time: 73.4ms (= 14689.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.2ms (= 17644.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 88.2ms / 73.4ms)

OneFlow resnet50 time: 64.5ms (= 12904.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.3ms (= 15051.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.17 (= 75.3ms / 64.5ms)

OneFlow resnet50 time: 56.1ms (= 11216.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 73.1ms (= 14617.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.30 (= 73.1ms / 56.1ms)

@github-actions
Copy link
Contributor

CI failed when running job: cuda-misc. PR label automerge has been removed

@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 19, 2022 04:28
@github-actions
Copy link
Contributor

Speed stats:

@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 19, 2022 08:52
@github-actions
Copy link
Contributor

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8124/

@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 130.0ms (= 13002.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 145.7ms (= 14570.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.12 (= 145.7ms / 130.0ms)

OneFlow resnet50 time: 77.6ms (= 7758.9ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.6ms (= 8459.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.09 (= 84.6ms / 77.6ms)

OneFlow resnet50 time: 52.8ms (= 10569.5ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 65.3ms (= 13050.5ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.23 (= 65.3ms / 52.8ms)

OneFlow resnet50 time: 43.1ms (= 8621.3ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 42.9ms (= 8580.5ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.00 (= 42.9ms / 43.1ms)

OneFlow resnet50 time: 35.8ms (= 7154.6ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 41.4ms (= 8282.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.16 (= 41.4ms / 35.8ms)

OneFlow swin dataloader time: 0.385s (= 77.081s / 200, num_workers=1)
PyTorch swin dataloader time: 0.153s (= 30.536s / 200, num_workers=1)
Relative speed: 0.396 (= 0.153s / 0.385s)

OneFlow swin dataloader time: 0.065s (= 13.051s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.212s / 200, num_workers=4)
Relative speed: 0.629 (= 0.041s / 0.065s)

OneFlow swin dataloader time: 0.035s (= 6.956s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.383s / 200, num_workers=8)
Relative speed: 0.630 (= 0.022s / 0.035s)

❌ OneFlow resnet50 time: 146.1ms (= 14608.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 168.8ms (= 16881.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 168.8ms / 146.1ms)

OneFlow resnet50 time: 95.2ms (= 9519.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 111.7ms (= 11170.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 111.7ms / 95.2ms)

OneFlow resnet50 time: 73.8ms (= 14755.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 98.7ms (= 19732.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.34 (= 98.7ms / 73.8ms)

OneFlow resnet50 time: 60.9ms (= 12177.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.1ms (= 15018.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.23 (= 75.1ms / 60.9ms)

OneFlow resnet50 time: 53.5ms (= 10709.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.7ms (= 15140.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.41 (= 75.7ms / 53.5ms)

@mosout mosout requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 19, 2022 14:05
@github-actions
Copy link
Contributor

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8124/

@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 130.1ms (= 13006.1ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.4ms (= 14337.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 143.4ms / 130.1ms)

OneFlow resnet50 time: 76.7ms (= 7665.2ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.2ms (= 8317.9ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.09 (= 83.2ms / 76.7ms)

OneFlow resnet50 time: 52.1ms (= 10420.6ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 64.8ms (= 12964.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.24 (= 64.8ms / 52.1ms)

OneFlow resnet50 time: 42.3ms (= 8459.3ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 42.2ms (= 8445.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.00 (= 42.2ms / 42.3ms)

OneFlow resnet50 time: 35.6ms (= 7118.6ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.8ms (= 7950.1ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.12 (= 39.8ms / 35.6ms)

OneFlow swin dataloader time: 0.248s (= 49.621s / 200, num_workers=1)
PyTorch swin dataloader time: 0.150s (= 30.053s / 200, num_workers=1)
Relative speed: 0.606 (= 0.150s / 0.248s)

OneFlow swin dataloader time: 0.064s (= 12.786s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.474s / 200, num_workers=4)
Relative speed: 0.663 (= 0.042s / 0.064s)

OneFlow swin dataloader time: 0.055s (= 11.092s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.454s / 200, num_workers=8)
Relative speed: 0.402 (= 0.022s / 0.055s)

❌ OneFlow resnet50 time: 145.6ms (= 14563.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 169.3ms (= 16931.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 169.3ms / 145.6ms)

OneFlow resnet50 time: 97.0ms (= 9701.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 111.9ms (= 11191.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 111.9ms / 97.0ms)

OneFlow resnet50 time: 74.0ms (= 14804.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.6ms (= 17725.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 88.6ms / 74.0ms)

OneFlow resnet50 time: 62.6ms (= 12513.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.7ms (= 14930.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 74.7ms / 62.6ms)

OneFlow resnet50 time: 54.6ms (= 10914.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.9ms (= 13988.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.28 (= 69.9ms / 54.6ms)

@mosout mosout merged commit c020673 into master May 19, 2022
@mosout mosout deleted the fix_cpp_api_mlir branch May 19, 2022 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jackalcooper jackalcooper jackalcooper approved these changes

@daquexian daquexian daquexian approved these changes

@hjchen2 hjchen2 Awaiting requested review from hjchen2 hjchen2 is a code owner

@BBuf BBuf Awaiting requested review from BBuf

@oneflow-ci-bot oneflow-ci-bot Awaiting requested review from oneflow-ci-bot

Assignees
No one assigned
Labels
automerge enhancement serving
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@mosout @jackalcooper @daquexian @BBuf @oneflow-ci-bot