Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimize global tensor err msg #7821

Merged
merged 16 commits into from
Mar 22, 2022
Merged

optimize global tensor err msg #7821

merged 16 commits into from
Mar 22, 2022

Conversation

Flowingsun007
Copy link
Contributor

@Flowingsun007 Flowingsun007 commented Mar 17, 2022 •
edited
Loading

@Flowingsun007 Flowingsun007 enabled auto-merge (squash) March 17, 2022 01:45
@Flowingsun007 Flowingsun007 added graph graph mode and removed bug labels Mar 17, 2022
@Flowingsun007 Flowingsun007 changed the title optimize globel tensor err msg optimize global tensor err msg Mar 18, 2022
@Flowingsun007 Flowingsun007 linked an issue Mar 18, 2022 that may be closed by this pull request
代码里面的网址链接404 #7643
Closed
@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.4ms (= 12837.2ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 138.9ms (= 13890.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.08 (= 138.9ms / 128.4ms)

❌ OneFlow resnet50 time: 79.1ms (= 7914.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 81.4ms (= 8138.6ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.03 (= 81.4ms / 79.1ms)

OneFlow resnet50 time: 50.3ms (= 10067.3ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.2ms (= 11635.8ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.16 (= 58.2ms / 50.3ms)

OneFlow resnet50 time: 41.8ms (= 8363.7ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 44.0ms (= 8802.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.05 (= 44.0ms / 41.8ms)

OneFlow resnet50 time: 38.1ms (= 7617.2ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 44.6ms (= 8916.7ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.17 (= 44.6ms / 38.1ms)

md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
build_dataset >>>>>> ImageFolder
build_dataset >>>>>> ImageFolder
✔️ Relative speed: 1.02 (= 50.3s / 51.0s)

md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
build_dataset >>>>>> ImageFolder
build_dataset >>>>>> ImageFolder
✔️ Relative speed: 1.05 (= 13.3s / 13.9s)

md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
build_dataset >>>>>> ImageFolder
build_dataset >>>>>> ImageFolder
✔️ Relative speed: 1.00 (= 7.4s / 7.4s)

✔️ OneFlow resnet50 time: 135.4ms (= 13544.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.0ms (= 16003.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 160.0ms / 135.4ms)

OneFlow resnet50 time: 86.3ms (= 8629.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.5ms (= 10153.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 101.5ms / 86.3ms)

OneFlow resnet50 time: 62.5ms (= 12498.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.2ms (= 15249.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 76.2ms / 62.5ms)

OneFlow resnet50 time: 53.5ms (= 10699.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 66.9ms (= 13377.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 66.9ms / 53.5ms)

OneFlow resnet50 time: 49.7ms (= 9945.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 62.6ms (= 12525.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 62.6ms / 49.7ms)

@Flowingsun007 Flowingsun007 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 18, 2022 08:16
@Flowingsun007 Flowingsun007 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 19, 2022 15:10
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 20, 2022 03:21
@github-actions
Copy link
Contributor

CI failed when running job: cuda-module-distributed-rank-1. PR label automerge has been removed

@Flowingsun007 Flowingsun007 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 21, 2022 06:44
@Flowingsun007 Flowingsun007 removed the request for review from oneflow-ci-bot March 22, 2022 12:46
@github-actions
Copy link
Contributor

Speed stats: 
GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.4ms (= 12844.2ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 146.5ms (= 14646.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.14 (= 146.5ms / 128.4ms)

✔️ OneFlow resnet50 time: 78.6ms (= 7863.6ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.0ms (= 8498.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.08 (= 85.0ms / 78.6ms)

OneFlow resnet50 time: 52.4ms (= 10473.3ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.2ms (= 11836.3ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.13 (= 59.2ms / 52.4ms)

OneFlow resnet50 time: 43.1ms (= 8625.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.4ms (= 9878.4ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.15 (= 49.4ms / 43.1ms)

OneFlow resnet50 time: 36.5ms (= 7295.1ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 41.8ms (= 8354.1ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.15 (= 41.8ms / 36.5ms)

md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
build_dataset >>>>>> ImageFolder
build_dataset >>>>>> ImageFolder
✔️ Relative speed: 1.02 (= 49.5s / 50.5s)

md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
build_dataset >>>>>> ImageFolder
build_dataset >>>>>> ImageFolder
✔️ Relative speed: 0.99 (= 13.4s / 13.2s)

md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
md5 /home/ci-user/ci-cache/test_cache/onerec_test/nanodataset.zip 7f5cde8b5a6c411107517ac9b00f29db
build_dataset >>>>>> ImageFolder
build_dataset >>>>>> ImageFolder
✔️ Relative speed: 0.97 (= 7.4s / 7.2s)

✔️ OneFlow resnet50 time: 135.6ms (= 13564.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 155.8ms (= 15576.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 155.8ms / 135.6ms)

OneFlow resnet50 time: 88.2ms (= 8823.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 106.7ms (= 10670.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 106.7ms / 88.2ms)

OneFlow resnet50 time: 61.8ms (= 12357.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.8ms (= 15556.2ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 77.8ms / 61.8ms)

OneFlow resnet50 time: 53.1ms (= 10627.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.0ms (= 13601.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.28 (= 68.0ms / 53.1ms)

OneFlow resnet50 time: 46.5ms (= 9307.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 61.8ms (= 12358.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.33 (= 61.8ms / 46.5ms)

@Flowingsun007 Flowingsun007 merged commit 1c5fb04 into master Mar 22, 2022
@Flowingsun007 Flowingsun007 deleted the optimize_tensor_error_msg branch March 22, 2022 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simonJJJ simonJJJ simonJJJ approved these changes

@BBuf BBuf BBuf approved these changes

@chengtbf chengtbf Awaiting requested review from chengtbf

@strint strint Awaiting requested review from strint strint is a code owner

@oneflow-ci-bot oneflow-ci-bot Awaiting requested review from oneflow-ci-bot

Assignees
No one assigned
Labels
api automerge documentation graph graph mode
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

代码里面的网址链接404
3 participants
@Flowingsun007 @simonJJJ @BBuf