implement addcmul op #7282

lcylcy · 2022-01-18T09:19:04Z

python/oneflow/__init__.py

EsdeathYZH · 2022-04-16T06:13:08Z

python/oneflow/framework/tensor.py

@@ -1151,6 +1156,8 @@ def RegisterMethods():
    Tensor.log2 = _log2
    Tensor.add = _add


我查了下torch.Tensor.add文档，我感觉目前oneflow.Tensor.add没有完全对齐（缺少了alpha参数）？

add 没对齐的问题和这个PR关系不大，不过确实是个问题，可以记录在这个 issue comment 里，会有人跟进的 https://github.com/Oneflow-Inc/OneTeam/issues/1207#issuecomment-1073432125

oneflow/core/functional/impl/binary_functor.cpp

oneflow/core/functional/impl/math_functor.cpp

doombeaker · 2022-04-17T14:43:39Z

这个 PR 涉及到 inplace functor 的实现，接口和 https://github.com/Oneflow-Inc/OneTeam/issues/806 这里好像已经有挺多不同了，我已经不熟悉了。

不知道 @wyg1997 能否抽空帮忙 review 下。谢谢～

python/oneflow/framework/docstr/math_ops.py

python/oneflow/framework/tensor.py

oneflow/core/functional/impl/binary_functor.cpp

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

wyg1997

还需要加上 eager global 单测，可以参考其他的 test_consistent_xxx.py 的写法。

本地 python -m oneflow.distributed.launch --nproc_per_node 4 test_consistent_addcmul.py 来测试

python/oneflow/framework/docstr/math_ops.py

github-actions · 2022-04-20T06:45:37Z

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

github-actions · 2022-04-20T12:32:01Z

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

github-actions · 2022-04-21T10:33:53Z

Speed stats:

GPU Name: GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.5ms (= 12953.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 147.3ms (= 14729.0ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.14 (= 147.3ms / 129.5ms)

OneFlow resnet50 time: 83.6ms (= 8364.7ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 86.8ms (= 8681.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.04 (= 86.8ms / 83.6ms)

OneFlow resnet50 time: 56.2ms (= 11234.0ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 60.5ms (= 12108.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.08 (= 60.5ms / 56.2ms)

OneFlow resnet50 time: 45.6ms (= 9128.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 42.9ms (= 8583.1ms / 200, input_shape=[2, 3, 224, 224])
❌ Relative speed: 0.94 (= 42.9ms / 45.6ms)

OneFlow resnet50 time: 40.2ms (= 8045.8ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 37.0ms (= 7399.8ms / 200, input_shape=[1, 3, 224, 224])
❌ Relative speed: 0.92 (= 37.0ms / 40.2ms)

OneFlow swin dataloader time: 0.255s (= 51.089s / 200, num_workers=1)
PyTorch swin dataloader time: 0.150s (= 30.069s / 200, num_workers=1)
Relative speed: 0.589 (= 0.150s / 0.255s)

OneFlow swin dataloader time: 0.069s (= 13.879s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.602s / 200, num_workers=4)
Relative speed: 0.620 (= 0.043s / 0.069s)

OneFlow swin dataloader time: 0.036s (= 7.294s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.335s / 200, num_workers=8)
Relative speed: 0.594 (= 0.022s / 0.036s)

✔️ OneFlow resnet50 time: 136.0ms (= 13601.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 165.9ms (= 16585.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 165.9ms / 136.0ms)

OneFlow resnet50 time: 92.6ms (= 9264.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 104.3ms (= 10428.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 104.3ms / 92.6ms)

OneFlow resnet50 time: 65.4ms (= 13076.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 79.0ms (= 15808.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 79.0ms / 65.4ms)

OneFlow resnet50 time: 54.0ms (= 10802.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 79.3ms (= 15856.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.47 (= 79.3ms / 54.0ms)

OneFlow resnet50 time: 50.4ms (= 10070.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.4ms (= 13086.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.30 (= 65.4ms / 50.4ms)

github-actions · 2022-04-21T10:37:04Z

CI failed when running job: cpu-module. PR label automerge has been removed

…_addcmul

github-actions · 2022-04-21T16:17:45Z

Speed stats:

github-actions · 2022-04-21T18:14:02Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7282/

github-actions · 2022-04-21T18:23:41Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.5ms (= 12847.9ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.1ms (= 14005.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.09 (= 140.1ms / 128.5ms)

OneFlow resnet50 time: 82.2ms (= 8217.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.2ms (= 8322.4ms / 100, input_shape=[8, 3, 224, 224])
❌ Relative speed: 1.01 (= 83.2ms / 82.2ms)

OneFlow resnet50 time: 54.1ms (= 10824.3ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 64.3ms (= 12864.7ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.19 (= 64.3ms / 54.1ms)

OneFlow resnet50 time: 45.9ms (= 9189.1ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 40.9ms (= 8180.4ms / 200, input_shape=[2, 3, 224, 224])
❌ Relative speed: 0.89 (= 40.9ms / 45.9ms)

OneFlow resnet50 time: 40.4ms (= 8071.6ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.1ms (= 7810.5ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.97 (= 39.1ms / 40.4ms)

OneFlow swin dataloader time: 0.258s (= 51.634s / 200, num_workers=1)
PyTorch swin dataloader time: 0.150s (= 30.097s / 200, num_workers=1)
Relative speed: 0.583 (= 0.150s / 0.258s)

OneFlow swin dataloader time: 0.068s (= 13.638s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.309s / 200, num_workers=4)
Relative speed: 0.609 (= 0.042s / 0.068s)

OneFlow swin dataloader time: 0.038s (= 7.532s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.462s / 200, num_workers=8)
Relative speed: 0.592 (= 0.022s / 0.038s)

✔️ OneFlow resnet50 time: 135.8ms (= 13584.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 166.0ms (= 16598.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 166.0ms / 135.8ms)

OneFlow resnet50 time: 90.7ms (= 9068.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 104.6ms (= 10458.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 104.6ms / 90.7ms)

OneFlow resnet50 time: 63.9ms (= 12784.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.7ms (= 15540.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 77.7ms / 63.9ms)

OneFlow resnet50 time: 53.3ms (= 10667.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.8ms (= 15369.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.44 (= 76.8ms / 53.3ms)

OneFlow resnet50 time: 50.9ms (= 10177.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.6ms (= 15310.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.50 (= 76.6ms / 50.9ms)

lcylcy requested review from doombeaker and BBuf January 18, 2022 09:19

lcylcy requested review from daquexian, hjchen2 and jackalcooper as code owners January 18, 2022 09:19

chengtbf mentioned this pull request Jan 24, 2022

implement select op #7320

Closed

lcylcy changed the title ~~firse version~~ addcmul_op Jan 24, 2022

EsdeathYZH changed the title ~~addcmul_op~~ implement cmul_op Apr 15, 2022

lcylcy added 2 commits April 15, 2022 20:06

firse version

0bba196

docs

46cfceb

EsdeathYZH force-pushed the lcy_addcmul branch from d21f3b5 to 46cfceb Compare April 15, 2022 12:08

EsdeathYZH added op enhancement labels Apr 15, 2022

EsdeathYZH reviewed Apr 15, 2022

View reviewed changes

python/oneflow/__init__.py Outdated Show resolved Hide resolved

fix doc

2ce1df1

EsdeathYZH changed the title ~~implement cmul_op~~ implement addcmul op Apr 16, 2022

EsdeathYZH added 2 commits April 16, 2022 13:57

reimpl functor

76b9f19

remove dup inplace checking

fa53179

EsdeathYZH reviewed Apr 16, 2022

View reviewed changes

oneflow/core/functional/impl/binary_functor.cpp Outdated Show resolved Hide resolved

EsdeathYZH added 2 commits April 16, 2022 14:27

refine

c5f0d4a

fix

8e46b8c

EsdeathYZH reviewed Apr 16, 2022

View reviewed changes

oneflow/core/functional/impl/math_functor.cpp Show resolved Hide resolved

Merge branch 'master' into lcy_addcmul

ff63381

doombeaker requested a review from wyg1997 April 17, 2022 14:44

doombeaker reviewed Apr 17, 2022

View reviewed changes

python/oneflow/framework/docstr/math_ops.py Show resolved Hide resolved

doombeaker reviewed Apr 17, 2022

View reviewed changes

python/oneflow/framework/tensor.py Outdated Show resolved Hide resolved

wyg1997 reviewed Apr 18, 2022

View reviewed changes

oneflow/core/functional/impl/binary_functor.cpp Outdated Show resolved Hide resolved

oneflow/core/functional/impl/binary_functor.cpp Outdated Show resolved Hide resolved

oneflow/core/functional/impl/binary_functor.cpp Outdated Show resolved Hide resolved

refine comment

71adb21

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

EsdeathYZH requested a review from wyg1997 April 18, 2022 09:46

wyg1997 reviewed Apr 18, 2022

View reviewed changes

python/oneflow/framework/docstr/math_ops.py Outdated Show resolved Hide resolved

EsdeathYZH added 2 commits April 19, 2022 15:30

refine docs

0244933

add global eager unit test

455dd50

EsdeathYZH requested a review from wyg1997 April 19, 2022 12:05

wyg1997 approved these changes Apr 20, 2022

View reviewed changes

EsdeathYZH added the automerge label Apr 20, 2022

Merge branch 'master' into lcy_addcmul

c65b799

doombeaker approved these changes Apr 20, 2022

View reviewed changes

doombeaker requested a review from oneflow-ci-bot April 20, 2022 06:44

auto format by CI

3f68cc2

mergify bot and others added 2 commits April 20, 2022 12:30

Merge branch 'master' into lcy_addcmul

70a42dc

auto format by CI

54f7320

mergify bot added 2 commits April 20, 2022 21:13

Merge branch 'master' into lcy_addcmul

a2c4b0a

Merge branch 'master' into lcy_addcmul

3f83eb4

github-actions bot removed the automerge label Apr 21, 2022

EsdeathYZH added 2 commits April 21, 2022 19:43

fix unit test

0094122

Merge branch 'lcy_addcmul' of github.com:Oneflow-Inc/oneflow into lcy…

0e31213

…_addcmul

EsdeathYZH added the automerge label Apr 21, 2022

mergify bot added 2 commits April 21, 2022 12:38

Merge branch 'master' into lcy_addcmul

da10424

Merge branch 'master' into lcy_addcmul

1b501d6

mergify bot merged commit 91b319d into master Apr 21, 2022

mergify bot deleted the lcy_addcmul branch April 21, 2022 19:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement addcmul op #7282

implement addcmul op #7282

lcylcy commented Jan 18, 2022 •

edited

Loading

EsdeathYZH Apr 16, 2022

doombeaker Apr 17, 2022 •

edited

Loading

doombeaker commented Apr 17, 2022

wyg1997 left a comment

github-actions bot commented Apr 20, 2022

github-actions bot commented Apr 20, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

		@@ -1151,6 +1156,8 @@ def RegisterMethods():
		Tensor.log2 = _log2
		Tensor.add = _add

implement addcmul op #7282

implement addcmul op #7282

Conversation

lcylcy commented Jan 18, 2022 • edited Loading

EsdeathYZH Apr 16, 2022

Choose a reason for hiding this comment

doombeaker Apr 17, 2022 • edited Loading

Choose a reason for hiding this comment

doombeaker commented Apr 17, 2022

wyg1997 left a comment

Choose a reason for hiding this comment

github-actions bot commented Apr 20, 2022

github-actions bot commented Apr 20, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

github-actions bot commented Apr 21, 2022

lcylcy commented Jan 18, 2022 •

edited

Loading

doombeaker Apr 17, 2022 •

edited

Loading