eager global sucessfully in 1card and 2card #7635

lcylcy · 2022-03-01T07:38:49Z

No description provided.

clackhan · 2022-03-01T08:20:37Z

python/oneflow/test/modules/test_consistent_norm.py

+
+@autotest(n=1, check_graph=False)
+def _test_tuple_dim_norm_with_random_data(test_case, placement, sbp):
+        input = random_tensor(ndim=2,dim0=8,dim1=8)


这里缩进应该是四个空格，调一下

clackhan · 2022-03-01T08:21:45Z

python/oneflow/test/modules/test_consistent_normalize.py

+from collections import OrderedDict
+
+import numpy as np
+from oneflow.test_utils.test_util import GenArgList


这些都没用，删除

clackhan · 2022-03-01T08:22:34Z

python/oneflow/test/modules/test_consistent_permute.py

+from collections import OrderedDict
+
+from random import shuffle
+import numpy as np
+from oneflow.test_utils.test_util import GenArgList


clackhan · 2022-03-01T08:25:39Z

python/oneflow/test/modules/test_consistent_pixel_shuffle.py

+    m = flow.nn.PixelShuffle(h_upscale_factor=h_upscale_factor, w_upscale_factor=w_upscale_factor)
+    m = m.to(device)
+    of_out = m(input)
+    np_out = _np_pixel_shuffle(x, h_upscale_factor, w_upscale_factor)


这里的x和input不一样，数值上就会有所不同

值为什么不一样

值为什么不一样

首先，x在每个rank上的值是不一样的，其次，input = flow.tensor(x).to_global(...)之后，如果是s(0)时，形状就会改变

liufengwei0103 · 2022-03-01T09:43:50Z

python/oneflow/test/modules/test_consistent_pixel_shuffle.py

+    m = m.to(device)
+    of_out = m(input)
+    np_out = _np_pixel_shuffle(x, h_upscale_factor, w_upscale_factor)
+    if flow.env.get_rank() == 0:


感觉所有rank都检查结果稳妥些

这个就是别的rank不检查结果吧

这个就是别的rank不检查结果吧

为啥？

好的，所以rank都检查吧

wyushun · 2022-03-01T14:28:22Z

python/oneflow/test/modules/test_consistent_norm.py

+import unittest
+from collections import OrderedDict
+
+import numpy as np
+from oneflow.test_utils.test_util import GenArgList


这些是不是都可以去掉，可以参考这个例子：https://github.com/Oneflow-Inc/oneflow/blob/master/python/oneflow/test/modules/test_consistent_greater_equal.py

wyushun · 2022-03-01T14:29:52Z

python/oneflow/test/modules/test_consistent_pixel_shuffle.py

+import unittest
+from collections import OrderedDict
+
+from random import shuffle
+import numpy as np
+from oneflow.test_utils.test_util import GenArgList


同上，这些import应该能删除一部分，你自己跑下看看哪些能删

wyushun · 2022-03-01T14:30:58Z

python/oneflow/test/modules/test_consistent_randperm.py

+import unittest
+from collections import OrderedDict
+
+from random import shuffle
+import numpy as np
+from oneflow.test_utils.test_util import GenArgList


clackhan · 2022-03-03T07:29:51Z

python/oneflow/test/modules/test_consistent_ones_like.py

+    y = flow.ones_like(x)
+    test_case.assertTrue(y.dtype is flow.float32)
+    test_case.assertTrue(y.shape == x.shape)
+    test_case.assertTrue(y.to_local().device == x.to_local().device)


这个地方也不太对，placement的类型type和device不一定一样，这里应该是过不了的

我跑没问题

clackhan · 2022-03-03T07:30:08Z

python/oneflow/test/modules/test_consistent_ones_like.py

+    y = flow.ones_like(x)
+    test_case.assertTrue(y.dtype is flow.int)
+    test_case.assertTrue(y.shape == x.shape)
+    test_case.assertTrue(y.to_local().device == x.to_local().device)


clackhan · 2022-03-03T07:34:10Z

python/oneflow/test/modules/test_consistent_pixel_shuffle.py

+    m = flow.nn.PixelShuffle(h_upscale_factor=h_upscale_factor, w_upscale_factor=w_upscale_factor)
+    m = m.to(device)
+    of_out = m(input)
+    np_out = _np_pixel_shuffle(x, h_upscale_factor, w_upscale_factor)


值为什么不一样

首先，x在每个rank上的值是不一样的，其次，input = flow.tensor(x).to_global(...)之后，如果是s(0)时，形状就会改变

clackhan · 2022-03-03T07:35:37Z

python/oneflow/test/modules/test_consistent_randperm.py

+
+if __name__ == "__main__":
+    unittest.main()
+


加空行，或则121行有空格，需要删除

clackhan · 2022-03-03T09:20:16Z

python/oneflow/test/modules/test_consistent_ones_like.py

+    test_case.assertTrue(y.dtype is flow.float32)
+    test_case.assertTrue(y.shape == x.shape)
+    test_case.assertTrue(y.placement == placement)
+    y_numpy = np.ones_like(x.numpy())


这里的y_numpy是不是应该通过y_numpy = np.ones(x.numpy(shape)) 获取更合适一点

clackhan · 2022-03-03T09:20:24Z

python/oneflow/test/modules/test_consistent_ones_like.py

+    test_case.assertTrue(y.dtype is flow.int)
+    test_case.assertTrue(y.shape == x.shape)
+    test_case.assertTrue(y.placement == placement)
+    y_numpy = np.ones_like(x.numpy())


github-actions · 2022-03-03T11:07:27Z

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

github-actions · 2022-03-03T13:20:06Z

CI failed when running job: cuda-module-distributed-rank-0. PR label automerge has been removed

github-actions · 2022-03-12T17:12:17Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 129.2ms (= 12918.1ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 145.6ms (= 14563.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.13 (= 145.6ms / 129.2ms)

❌ OneFlow resnet50 time: 80.1ms (= 8014.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.7ms (= 8771.9ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.09 (= 87.7ms / 80.1ms)

OneFlow resnet50 time: 53.2ms (= 10649.6ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 55.5ms (= 11106.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.04 (= 55.5ms / 53.2ms)

OneFlow resnet50 time: 44.8ms (= 8967.0ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.7ms (= 9939.0ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.11 (= 49.7ms / 44.8ms)

OneFlow resnet50 time: 39.6ms (= 7918.2ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 38.9ms (= 7786.9ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.98 (= 38.9ms / 39.6ms)

✔️ OneFlow resnet50 time: 142.0ms (= 14196.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.1ms (= 16310.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 163.1ms / 142.0ms)

OneFlow resnet50 time: 91.0ms (= 9099.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 103.0ms (= 10298.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 103.0ms / 91.0ms)

OneFlow resnet50 time: 62.3ms (= 12468.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.2ms (= 15440.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.24 (= 77.2ms / 62.3ms)

OneFlow resnet50 time: 51.7ms (= 10336.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 66.8ms (= 13355.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.29 (= 66.8ms / 51.7ms)

OneFlow resnet50 time: 50.4ms (= 10070.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 62.4ms (= 12472.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.24 (= 62.4ms / 50.4ms)

github-actions · 2022-03-12T17:12:18Z

CI failed when running job: cuda-speed-test. PR label automerge has been removed

github-actions · 2022-03-12T21:20:56Z

Speed stats:

github-actions · 2022-03-12T21:26:04Z

CI failed when running job: cpu-module. PR label automerge has been removed

github-actions · 2022-03-12T21:26:33Z

Speed stats:

github-actions · 2022-03-13T00:06:26Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.8ms (= 12879.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.7ms (= 14071.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.09 (= 140.7ms / 128.8ms)

✔️ OneFlow resnet50 time: 78.2ms (= 7817.7ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 88.2ms (= 8822.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.13 (= 88.2ms / 78.2ms)

OneFlow resnet50 time: 53.0ms (= 10607.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 54.6ms (= 10927.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.03 (= 54.6ms / 53.0ms)

OneFlow resnet50 time: 45.2ms (= 9035.8ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.5ms (= 9907.0ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.10 (= 49.5ms / 45.2ms)

OneFlow resnet50 time: 38.3ms (= 7665.1ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.4ms (= 7876.5ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.03 (= 39.4ms / 38.3ms)

✔️ OneFlow resnet50 time: 142.8ms (= 14279.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 168.1ms (= 16812.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 168.1ms / 142.8ms)

OneFlow resnet50 time: 88.6ms (= 8864.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 107.8ms (= 10782.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 107.8ms / 88.6ms)

OneFlow resnet50 time: 61.6ms (= 12328.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.5ms (= 15290.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.24 (= 76.5ms / 61.6ms)

OneFlow resnet50 time: 54.1ms (= 10825.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 63.5ms (= 12698.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 63.5ms / 54.1ms)

OneFlow resnet50 time: 47.1ms (= 9413.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 62.1ms (= 12410.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.32 (= 62.1ms / 47.1ms)

github-actions · 2022-03-13T00:35:24Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.9ms (= 12890.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 139.4ms (= 13941.6ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.08 (= 139.4ms / 128.9ms)

✔️ OneFlow resnet50 time: 78.1ms (= 7809.6ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.3ms (= 8729.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.12 (= 87.3ms / 78.1ms)

OneFlow resnet50 time: 53.4ms (= 10688.0ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 60.7ms (= 12145.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.14 (= 60.7ms / 53.4ms)

OneFlow resnet50 time: 44.3ms (= 8862.5ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.6ms (= 9914.6ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.12 (= 49.6ms / 44.3ms)

OneFlow resnet50 time: 39.4ms (= 7875.0ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 44.5ms (= 8905.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.13 (= 44.5ms / 39.4ms)

✔️ OneFlow resnet50 time: 142.6ms (= 14263.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.0ms (= 16304.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.14 (= 163.0ms / 142.6ms)

OneFlow resnet50 time: 90.7ms (= 9073.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 107.1ms (= 10709.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 107.1ms / 90.7ms)

OneFlow resnet50 time: 62.6ms (= 12524.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.5ms (= 15494.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.24 (= 77.5ms / 62.6ms)

OneFlow resnet50 time: 49.4ms (= 9879.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 67.4ms (= 13477.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.36 (= 67.4ms / 49.4ms)

OneFlow resnet50 time: 47.0ms (= 9398.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 62.3ms (= 12453.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.33 (= 62.3ms / 47.0ms)

* eager global sucessfully in 1card and 2card * remove some import * revise pixel_shuffle, ones_like * delete randperm * using ones * auto format by CI * test to _test * eager global sucessfully in 1card and 2card (#7635) eager global sucessfully in 1card and 2card (#7635) eager global sucessfully in 1card and 2card (#7635) eager global sucessfully in 1card and 2card (#7635) eager global sucessfully in 1card and 2card * remove some import * revise pixel_shuffle, ones_like * delete randperm * using ones * Merge branch 'master' into lch_eager_succeed * auto format by CI * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * test to _test * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * Merge branch 'master' into lch_eager_succeed * rm test pixel shuffle Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org> Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com> Co-authored-by: mergify-bot <noreply@mergify.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

eager global sucessfully in 1card and 2card

b8d6799

lcylcy requested review from daquexian and jackalcooper as code owners March 1, 2022 07:38

clackhan reviewed Mar 1, 2022

View reviewed changes

liufengwei0103 reviewed Mar 1, 2022

View reviewed changes

wyushun reviewed Mar 1, 2022

View reviewed changes

remove some import

a7dd594

clackhan reviewed Mar 3, 2022

View reviewed changes

lcylcy added 2 commits March 3, 2022 16:08

revise pixel_shuffle, ones_like

60ab97c

delete randperm

047aa98

lcylcy force-pushed the lch_eager_succeed branch from b9aa893 to 047aa98 Compare March 3, 2022 08:51

clackhan approved these changes Mar 3, 2022

View reviewed changes

liufengwei0103 approved these changes Mar 3, 2022

View reviewed changes

lcylcy and others added 2 commits March 3, 2022 19:04

using ones

b9d2cb7

Merge branch 'master' into lch_eager_succeed

3e4fb49

lcylcy added automerge ci python test labels Mar 3, 2022

lcylcy requested a review from oneflow-ci-bot March 3, 2022 11:06

auto format by CI

a3fe595

lcylcy requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 3, 2022 11:09

oneflow-ci-bot removed their request for review March 3, 2022 11:20

Merge branch 'master' into lch_eager_succeed

0099c74

oneflow-ci-bot self-requested a review March 3, 2022 12:49

github-actions bot removed the automerge label Mar 12, 2022

lcylcy removed the request for review from oneflow-ci-bot March 12, 2022 16:25

lcylcy added the automerge label Mar 12, 2022

lcylcy requested a review from oneflow-ci-bot March 12, 2022 16:25

github-actions bot removed the automerge label Mar 12, 2022

lcylcy added the automerge label Mar 12, 2022

lcylcy requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 12, 2022 17:24

Merge branch 'master' into lch_eager_succeed

82579f8

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 12, 2022 20:47

github-actions bot removed the automerge label Mar 12, 2022

lcylcy added the automerge label Mar 12, 2022

lcylcy requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 12, 2022 23:56

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 13, 2022 00:09

mergify bot merged commit 1a2383f into master Mar 13, 2022

mergify bot deleted the lch_eager_succeed branch March 13, 2022 00:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eager global sucessfully in 1card and 2card #7635

eager global sucessfully in 1card and 2card #7635

lcylcy commented Mar 1, 2022

clackhan Mar 1, 2022

clackhan Mar 1, 2022

clackhan Mar 1, 2022

clackhan Mar 1, 2022

lcylcy Mar 1, 2022

clackhan Mar 3, 2022

liufengwei0103 Mar 1, 2022 •

edited

Loading

lcylcy Mar 1, 2022

liufengwei0103 Mar 1, 2022

lcylcy Mar 1, 2022 •

edited

Loading

wyushun Mar 1, 2022

wyushun Mar 1, 2022 •

edited

Loading

wyushun Mar 1, 2022

clackhan Mar 3, 2022

lcylcy Mar 3, 2022

clackhan Mar 3, 2022

clackhan Mar 3, 2022

clackhan Mar 3, 2022

clackhan Mar 3, 2022

clackhan Mar 3, 2022

github-actions bot commented Mar 3, 2022

github-actions bot commented Mar 3, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 13, 2022

github-actions bot commented Mar 13, 2022

eager global sucessfully in 1card and 2card #7635

eager global sucessfully in 1card and 2card #7635

Conversation

lcylcy commented Mar 1, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liufengwei0103 Mar 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lcylcy Mar 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wyushun Mar 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 3, 2022

github-actions bot commented Mar 3, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 12, 2022

github-actions bot commented Mar 13, 2022

github-actions bot commented Mar 13, 2022

liufengwei0103 Mar 1, 2022 •

edited

Loading

lcylcy Mar 1, 2022 •

edited

Loading

wyushun Mar 1, 2022 •

edited

Loading