Dev tril op #6005

Ikkyu321 · 2021-08-23T04:20:00Z

对齐pytorch.tril算子
求单个或者批量2-D张量延相对主对角线位移指定diagnal的下三角张量

…dev_tril_op

hengzi · 2021-08-23T06:53:05Z

oneflow/core/autograd/gradient_funcs/tril.cpp

+namespace oneflow {
+namespace one {
+
+struct TrilInterpState : public AutoGradCaptureState {


TrilInterpState 改为 TrilCaptureState

TrilInterpState 改为 TrilCaptureState

已修改

hengzi · 2021-08-23T06:56:00Z

python/oneflow/nn/modules/tril.py

+        tensor([[1., 0., 0.],
+                [1., 1., 0.],
+                [1., 1., 1.]], dtype=oneflow.float32)
+        >>> x.tril(1)


这里只保留 flow.tril 的例子吧

这里只保留 flow.tril 的例子吧

已修改

…into dev_tril_op

Flowingsun007 · 2021-08-23T09:19:47Z

python/oneflow/test/modules/test_tril.py

+        return y
+
+    @autotest()
+    def TestTril_with_diag(test_case):


这个TestTril用小写吧（保持和上面的对齐）

这个TestTril用小写吧（保持和上面的对齐）

已修改

oneflow/core/functional/impl/array_functor.cpp

github-actions · 2021-08-23T10:44:16Z

CI failed, removing label automerge

github-actions · 2021-08-23T12:12:29Z

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.4ms (= 7020.1ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.6ms (= 6427.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 140.4ms / 128.6ms)

PyTorch resnet50 time: 84.1ms (= 4203.6ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.7ms (= 3735.4ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 84.1ms / 74.7ms)

PyTorch resnet50 time: 56.3ms (= 2816.8ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.2ms (= 2359.9ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.19 (= 56.3ms / 47.2ms)

PyTorch resnet50 time: 48.9ms (= 2444.1ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 37.5ms (= 1875.8ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.30 (= 48.9ms / 37.5ms)

PyTorch resnet50 time: 41.8ms (= 2088.6ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 43.5ms (= 2176.8ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 0.96 (= 41.8ms / 43.5ms)

github-actions · 2021-08-23T12:42:01Z

CI failed, removing label automerge

github-actions · 2021-08-23T13:22:56Z

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 141.4ms (= 7068.0ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.4ms (= 6422.3ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 141.4ms / 128.4ms)

PyTorch resnet50 time: 84.6ms (= 4231.2ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.7ms (= 3733.9ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 84.6ms / 74.7ms)

PyTorch resnet50 time: 57.9ms (= 2893.4ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.8ms (= 2389.6ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.21 (= 57.9ms / 47.8ms)

PyTorch resnet50 time: 53.2ms (= 2659.8ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 39.7ms (= 1984.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.34 (= 53.2ms / 39.7ms)

PyTorch resnet50 time: 44.6ms (= 2228.6ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 39.0ms (= 1951.8ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 44.6ms / 39.0ms)

github-actions · 2021-08-24T13:53:01Z

CI failed, removing label automerge

github-actions · 2021-08-24T13:53:06Z

CI failed, removing label automerge

github-actions · 2021-08-25T09:34:43Z

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 139.6ms (= 6977.9ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 127.8ms (= 6390.8ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 139.6ms / 127.8ms)

PyTorch resnet50 time: 82.3ms (= 4116.5ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.3ms (= 3716.6ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.11 (= 82.3ms / 74.3ms)

PyTorch resnet50 time: 60.3ms (= 3016.1ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 49.0ms (= 2451.7ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.23 (= 60.3ms / 49.0ms)

PyTorch resnet50 time: 47.7ms (= 2385.2ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 40.0ms (= 2002.5ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.19 (= 47.7ms / 40.0ms)

PyTorch resnet50 time: 43.9ms (= 2195.1ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 36.8ms (= 1839.0ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.19 (= 43.9ms / 36.8ms)

Ikkyu321 added 4 commits August 20, 2021 19:34

add tril op

7c46d0b

mod tril gradfunc

b010b9c

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …

4a2acba

…dev_tril_op

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …

99b642c

…dev_tril_op

hengzi reviewed Aug 23, 2021

View reviewed changes

Ikkyu321 added api eager enhancement op labels Aug 23, 2021

Merge branch 'master' into dev_tril_op

e67afbd

Ikkyu321 marked this pull request as ready for review August 23, 2021 08:40

Ikkyu321 added 2 commits August 23, 2021 16:59

Merge branch 'dev_tril_op' of https://github.com/Oneflow-Inc/oneflow …

c2a18a6

…into dev_tril_op

add tril test, resolve review bugs

e2e2c1f

Flowingsun007 reviewed Aug 23, 2021

View reviewed changes

oneflow/core/functional/impl/array_functor.cpp Show resolved Hide resolved

Flowingsun007 approved these changes Aug 23, 2021

View reviewed changes

mod tril test

546d0bb

Ikkyu321 added the automerge label Aug 23, 2021

Merge branch 'master' into dev_tril_op

65a6a08

oneflow-ci-bot self-requested a review August 23, 2021 10:22

github-actions bot removed the automerge label Aug 23, 2021

Merge branch 'master' into dev_tril_op

20fe3bc

Flowingsun007 added the automerge label Aug 23, 2021

Flowingsun007 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 23, 2021 12:20

github-actions bot removed the automerge label Aug 23, 2021

Ikkyu321 added the automerge label Aug 24, 2021

Ikkyu321 and others added 2 commits August 24, 2021 15:02

Merge branch 'master' into dev_tril_op

c001fb6

Merge branch 'master' into dev_tril_op

40b534d

oneflow-ci-bot self-requested a review August 24, 2021 07:02

Merge branch 'master' into dev_tril_op

7f49cf4

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 24, 2021 10:25

Merge branch 'master' into dev_tril_op

68f11d2

oneflow-ci-bot self-requested a review August 24, 2021 11:33

github-actions bot removed the automerge label Aug 24, 2021

oneflow-ci-bot removed their request for review August 24, 2021 13:55

hengzi approved these changes Aug 25, 2021

View reviewed changes

hengzi added the automerge label Aug 25, 2021

oneflow-ci-bot added 2 commits August 25, 2021 10:41

Merge branch 'master' into dev_tril_op

760eea3

Merge branch 'master' into dev_tril_op

288f5aa

oneflow-ci-bot self-requested a review August 25, 2021 05:20

Merge branch 'master' into dev_tril_op

e99df48

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 25, 2021 06:30

Merge branch 'master' into dev_tril_op

fad0b93

oneflow-ci-bot self-requested a review August 25, 2021 07:59

oneflow-ci-bot removed their request for review August 25, 2021 09:36

oneflow-ci-bot merged commit 09044ea into master Aug 25, 2021

oneflow-ci-bot deleted the dev_tril_op branch August 25, 2021 09:38

BBuf mentioned this pull request Sep 28, 2021

OneFlow 算子对齐 PyTorch 完备计划推进表 #4936

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev tril op #6005

Dev tril op #6005

Ikkyu321 commented Aug 23, 2021 •

edited

Loading

hengzi Aug 23, 2021

Ikkyu321 Aug 23, 2021

hengzi Aug 23, 2021

Ikkyu321 Aug 23, 2021

Flowingsun007 Aug 23, 2021

Ikkyu321 Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 24, 2021

github-actions bot commented Aug 24, 2021

github-actions bot commented Aug 25, 2021

Dev tril op #6005

Dev tril op #6005

Conversation

Ikkyu321 commented Aug 23, 2021 • edited Loading

hengzi Aug 23, 2021

Choose a reason for hiding this comment

Ikkyu321 Aug 23, 2021

Choose a reason for hiding this comment

hengzi Aug 23, 2021

Choose a reason for hiding this comment

Ikkyu321 Aug 23, 2021

Choose a reason for hiding this comment

Flowingsun007 Aug 23, 2021

Choose a reason for hiding this comment

Ikkyu321 Aug 23, 2021

Choose a reason for hiding this comment

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 23, 2021

github-actions bot commented Aug 24, 2021

github-actions bot commented Aug 24, 2021

github-actions bot commented Aug 25, 2021

Ikkyu321 commented Aug 23, 2021 •

edited

Loading