Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refine StreamContext #6191

Merged
merged 7 commits into from
Sep 8, 2021
Merged

Refine StreamContext #6191

merged 7 commits into from
Sep 8, 2021

Conversation

liujuncheng
Copy link
Collaborator

No description provided.

@liujuncheng liujuncheng requested a review from lixinqi September 7, 2021 08:05
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 7, 2021 08:33
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 7, 2021 09:38
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2021

Speed stats: 
GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.8ms (= 6439.8ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.4ms (= 7021.8ms / 50, input_shape=[16, 3, 224, 224])
Relative speed: 1.09 (= 140.4ms / 128.8ms)

OneFlow resnet50 time: 74.8ms (= 3739.9ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 86.2ms (= 4310.0ms / 50, input_shape=[8, 3, 224, 224])
Relative speed: 1.15 (= 86.2ms / 74.8ms)

OneFlow resnet50 time: 48.8ms (= 2438.3ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.2ms (= 2960.6ms / 50, input_shape=[4, 3, 224, 224])
Relative speed: 1.21 (= 59.2ms / 48.8ms)

OneFlow resnet50 time: 39.8ms (= 1992.3ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 48.2ms (= 2410.4ms / 50, input_shape=[2, 3, 224, 224])
Relative speed: 1.21 (= 48.2ms / 39.8ms)

OneFlow resnet50 time: 34.9ms (= 1747.4ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 42.6ms (= 2130.6ms / 50, input_shape=[1, 3, 224, 224])
Relative speed: 1.22 (= 42.6ms / 34.9ms)

OneFlow resnet50 time: 154.9ms (= 7743.0ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 162.5ms (= 8125.5ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
Relative speed: 1.05 (= 162.5ms / 154.9ms)

OneFlow resnet50 time: 102.7ms (= 5135.3ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 114.0ms (= 5700.9ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
Relative speed: 1.11 (= 114.0ms / 102.7ms)

OneFlow resnet50 time: 80.9ms (= 4043.8ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.8ms (= 3739.8ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
Relative speed: 0.92 (= 74.8ms / 80.9ms)

OneFlow resnet50 time: 78.2ms (= 3910.2ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.4ms (= 3470.0ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
Relative speed: 0.89 (= 69.4ms / 78.2ms)

OneFlow resnet50 time: 73.2ms (= 3662.4ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 63.9ms (= 3194.9ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
Relative speed: 0.87 (= 63.9ms / 73.2ms)

@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2021

CI failed, removing label automerge

@github-actions github-actions bot removed the automerge label Sep 7, 2021
@oneflow-ci-bot oneflow-ci-bot removed their request for review September 7, 2021 11:25
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2021

CI failed, removing label automerge

@github-actions github-actions bot removed the automerge label Sep 7, 2021
@oneflow-ci-bot oneflow-ci-bot removed their request for review September 7, 2021 13:10
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 7, 2021 14:08
@oneflow-ci-bot oneflow-ci-bot self-requested a review September 7, 2021 14:47
@oneflow-ci-bot oneflow-ci-bot removed their request for review September 7, 2021 16:00
@oneflow-ci-bot oneflow-ci-bot self-requested a review September 7, 2021 16:00
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2021

CI failed, removing label automerge

@github-actions github-actions bot removed the automerge label Sep 7, 2021
@oneflow-ci-bot oneflow-ci-bot removed their request for review September 7, 2021 17:48
@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2021

CI failed, removing label automerge

@github-actions github-actions bot removed the automerge label Sep 8, 2021
@oneflow-ci-bot oneflow-ci-bot removed their request for review September 8, 2021 01:33
@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2021

Speed stats: 
GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.4ms (= 6418.5ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.4ms (= 7020.5ms / 50, input_shape=[16, 3, 224, 224])
Relative speed: 1.09 (= 140.4ms / 128.4ms)

OneFlow resnet50 time: 74.8ms (= 3739.0ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.8ms (= 4241.7ms / 50, input_shape=[8, 3, 224, 224])
Relative speed: 1.13 (= 84.8ms / 74.8ms)

OneFlow resnet50 time: 47.6ms (= 2381.4ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 62.1ms (= 3103.2ms / 50, input_shape=[4, 3, 224, 224])
Relative speed: 1.30 (= 62.1ms / 47.6ms)

OneFlow resnet50 time: 39.2ms (= 1959.2ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 45.2ms (= 2257.6ms / 50, input_shape=[2, 3, 224, 224])
Relative speed: 1.15 (= 45.2ms / 39.2ms)

OneFlow resnet50 time: 43.5ms (= 2172.5ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.5ms (= 1975.4ms / 50, input_shape=[1, 3, 224, 224])
Relative speed: 0.91 (= 39.5ms / 43.5ms)

OneFlow resnet50 time: 153.5ms (= 7673.1ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 158.5ms (= 7925.1ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
Relative speed: 1.03 (= 158.5ms / 153.5ms)

OneFlow resnet50 time: 102.0ms (= 5101.7ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 99.6ms (= 4977.9ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
Relative speed: 0.98 (= 99.6ms / 102.0ms)

OneFlow resnet50 time: 76.3ms (= 3813.8ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 72.7ms (= 3634.3ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
Relative speed: 0.95 (= 72.7ms / 76.3ms)

OneFlow resnet50 time: 67.1ms (= 3352.8ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 63.4ms (= 3168.9ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
Relative speed: 0.95 (= 63.4ms / 67.1ms)

OneFlow resnet50 time: 70.5ms (= 3522.5ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 63.0ms (= 3149.3ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
Relative speed: 0.89 (= 63.0ms / 70.5ms)

@oneflow-ci-bot oneflow-ci-bot merged commit a0ba3b0 into master Sep 8, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the dev_refine_stream_context branch September 8, 2021 02:21
strint added a commit that referenced this pull request Sep 9, 2021
@strint @liujuncheng
@oneflow-ci-bot @clackhan @chengtbf @wyg1997 @Ikkyu321 @hengzi @PragmaTwice @daquexian @MARD1NO @Flowingsun007 @BBuf @hjchen2
Fea/checkpoint stop identity (#6216)
716cc69 
* Primitive (#6183)

* Add Primitive

* #ifdef WITH_CUDA

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Disable implicit boxing when parallel num eq one (#6188)

* mv_boxing_folder_to_core

* minor fix

* disable_implicit_boxing_when_parallel_num_eq_one

* Update eager_consistent_op_interpreter.cpp

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Lazy support Scalar (#6181)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Fix LayerNorm check bug (#6196)

* fix(Layernorm): fix check bug

* fix judge whether cpu or not

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add glu op (#6065)

* add glu op

* del glu_op export，align with torch

* mod glu_op

* mov op logic to C++

* Solve problems

* solve conflict

* delete gradient functor

* add ndim check

* add GLU test

* delete blank line

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>

* Primitive based copy task node (#6195)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* KernelState (#6198)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* container_util: fix VectorAt, remove useless MutMapAt (#6172)

* fcontainer_util: fix VectorAt, remove useless MutMapAt

* fcontainer_util: format

* MapAt: add default value version

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Refine StreamContext (#6191)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu symetric s to s (#6153)

* mv_boxing_folder_to_core

* minor fix

* cpu_symetric_s_to_s

* add test case

* auto format by CI

* minor fix

* refine

* Update eager_nccl_kernels.cpp

* minor fix

* fix bug

* minor fix

* Update oneflow/user/kernels/eager_nccl_kernels.cpp

Co-authored-by: daquexian <daquexian566@gmail.com>

* Update eager_nccl_kernels.cpp

* Update eager_nccl_kernels.cpp

* minor fix

* Update eager_nccl_kernels.cpp

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>

* fix bug (#6197)

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix consistent tensor zeros (#6202)

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* [Feat.] nn.Graph support grad acc with input/output tensor (#6155)

* nn.Graph support grad acc with input/output tensor

* dirty pass grad acc

* revert tensor.backward hack

* fix indent

* default S0 -> B

* Pack op/kernel support scalar input

* nn.Graph output pack support loss scalar

* add test script

* pass test

* Lazy build output eager tensors after job complete

* non scalar output test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Dev eliminate gcc warnings (#6199)

* fix gcc warning

* refine

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* StreamContextAdapter (#6205)

* StreamContextAdapter

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Autotest generate input tensor (#6206)

* Add tensor yaml, support export tensor functional api.

* refine

* Remove packed functor signature

* remove unused file

* Refine

* refine

* add activation op import

* reinit oneflow init.py

* add oneflow abs and exp

* add oneflow abs and exp

* add acos

* add arccosh

* add more op

* add more ops

* add more op

* add more ops

* add log1p

* add more smaples

* add more ops

* add more ops

* add more ops

* add more ops

* Complete tensor functional apis.

* Fix pybind call

* add more ops

* add ops done

* Add target of_functional_tensor_obj

* Disable throw visibility warnings

* fix target link

* fix

* fix incorrect use of flow.Tensor.

* Fix error merge

* fix

* fix add unittest

* refine

* refine

* fix

* fix

* add tensor doc

* auto format by CI

* refine

* Fix

* Add doc for python function

* refine

* add tensor method docstring

* fix some bug

* fix docs bug

* Fix

* auto format by CI

* Tensor->tensor

* Tensor->tensor

* refine Tensor->tensor

* fix

* fix

* fix

* fix conflict

* fix bug

* fix ci bug

* fix

* delete diag op

* fix conflict

* Fix segment

* fix

* merge

* autotest framework generate input tensor

* autotest framework generate input tensor

* fix bug

* fix impl bug

* refine

* refine

* refine

* fix

* fix

* fix comments

* delete useless

* fix ci error

* fix ci error

Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cleanup KernelUtil (#6212)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Rename flow to oneflow in user hint (#6190)

* style(*): rename flow to oneflow in user hint

* fix(*): fix doctest

* auto format by CI

* remove ddp speed test

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>

* merg and refactor

* refact code

* add io identity for activation checkpointing

Co-authored-by: Juncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: cheng cheng <472491134@qq.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: QiangX-man <87475073+QiangX-man@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: ZZK <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
oneflow-ci-bot added a commit that referenced this pull request Sep 10, 2021
@tingkuanpei @strint
@liujuncheng @oneflow-ci-bot @clackhan @chengtbf @wyg1997 @Ikkyu321 @hengzi @PragmaTwice @daquexian @MARD1NO @Flowingsun007 @BBuf @hjchen2
graph activation checkpointing (#6192)
1f354f1 
* Add test_graph_activation_checkpoint.py

* Fea/checkpoint stop identity (#6216)

* Primitive (#6183)

* Add Primitive

* #ifdef WITH_CUDA

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Disable implicit boxing when parallel num eq one (#6188)

* mv_boxing_folder_to_core

* minor fix

* disable_implicit_boxing_when_parallel_num_eq_one

* Update eager_consistent_op_interpreter.cpp

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Lazy support Scalar (#6181)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Fix LayerNorm check bug (#6196)

* fix(Layernorm): fix check bug

* fix judge whether cpu or not

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add glu op (#6065)

* add glu op

* del glu_op export，align with torch

* mod glu_op

* mov op logic to C++

* Solve problems

* solve conflict

* delete gradient functor

* add ndim check

* add GLU test

* delete blank line

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>

* Primitive based copy task node (#6195)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* KernelState (#6198)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* container_util: fix VectorAt, remove useless MutMapAt (#6172)

* fcontainer_util: fix VectorAt, remove useless MutMapAt

* fcontainer_util: format

* MapAt: add default value version

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Refine StreamContext (#6191)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu symetric s to s (#6153)

* mv_boxing_folder_to_core

* minor fix

* cpu_symetric_s_to_s

* add test case

* auto format by CI

* minor fix

* refine

* Update eager_nccl_kernels.cpp

* minor fix

* fix bug

* minor fix

* Update oneflow/user/kernels/eager_nccl_kernels.cpp

Co-authored-by: daquexian <daquexian566@gmail.com>

* Update eager_nccl_kernels.cpp

* Update eager_nccl_kernels.cpp

* minor fix

* Update eager_nccl_kernels.cpp

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>

* fix bug (#6197)

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix consistent tensor zeros (#6202)

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* [Feat.] nn.Graph support grad acc with input/output tensor (#6155)

* nn.Graph support grad acc with input/output tensor

* dirty pass grad acc

* revert tensor.backward hack

* fix indent

* default S0 -> B

* Pack op/kernel support scalar input

* nn.Graph output pack support loss scalar

* add test script

* pass test

* Lazy build output eager tensors after job complete

* non scalar output test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Dev eliminate gcc warnings (#6199)

* fix gcc warning

* refine

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* StreamContextAdapter (#6205)

* StreamContextAdapter

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Autotest generate input tensor (#6206)

* Add tensor yaml, support export tensor functional api.

* refine

* Remove packed functor signature

* remove unused file

* Refine

* refine

* add activation op import

* reinit oneflow init.py

* add oneflow abs and exp

* add oneflow abs and exp

* add acos

* add arccosh

* add more op

* add more ops

* add more op

* add more ops

* add log1p

* add more smaples

* add more ops

* add more ops

* add more ops

* add more ops

* Complete tensor functional apis.

* Fix pybind call

* add more ops

* add ops done

* Add target of_functional_tensor_obj

* Disable throw visibility warnings

* fix target link

* fix

* fix incorrect use of flow.Tensor.

* Fix error merge

* fix

* fix add unittest

* refine

* refine

* fix

* fix

* add tensor doc

* auto format by CI

* refine

* Fix

* Add doc for python function

* refine

* add tensor method docstring

* fix some bug

* fix docs bug

* Fix

* auto format by CI

* Tensor->tensor

* Tensor->tensor

* refine Tensor->tensor

* fix

* fix

* fix

* fix conflict

* fix bug

* fix ci bug

* fix

* delete diag op

* fix conflict

* Fix segment

* fix

* merge

* autotest framework generate input tensor

* autotest framework generate input tensor

* fix bug

* fix impl bug

* refine

* refine

* refine

* fix

* fix

* fix comments

* delete useless

* fix ci error

* fix ci error

Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cleanup KernelUtil (#6212)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Rename flow to oneflow in user hint (#6190)

* style(*): rename flow to oneflow in user hint

* fix(*): fix doctest

* auto format by CI

* remove ddp speed test

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>

* merg and refactor

* refact code

* add io identity for activation checkpointing

Co-authored-by: Juncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: cheng cheng <472491134@qq.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: QiangX-man <87475073+QiangX-man@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: ZZK <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>

* rm useless code

* refien test

* add skip

* import os

* fix test

* auto format by CI

Co-authored-by: strint <xiaoyulink@gmail.com>
Co-authored-by: Juncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: cheng cheng <472491134@qq.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: QiangX-man <87475073+QiangX-man@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: ZZK <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
oneflow-ci-bot added a commit that referenced this pull request Sep 12, 2021
@strint @chengtbf
@tingkuanpei @oneflow-ci-bot @liujuncheng @clackhan @wyg1997 @Ikkyu321 @hengzi @PragmaTwice @daquexian @MARD1NO @Flowingsun007 @BBuf @hjchen2
Fea/pipeline in graph (#6105)
1b901ee 
* add demo

* add test draft

* debug pipeline graph

* add pp

* add

* support 0-d tensor spilt

* add repr of conf for debug

* add acc grad

* add tag

* hack stage id of to_consistent

* Pipeline buffer pass stage id check log

* FixPipelineStageIdPass interface

* nn.Graph support grad acc with input/output tensor

* dirty pass grad acc

* Lazy support Scalar

* add log

* revert tensor.backward hack

* fix indent

* default S0 -> B

* merge

* Pack op/kernel support scalar input

* nn.Graph output pack support loss scalar

* add test script

* add pack

* pass test

* Add test_graph_activation_checkpoint.py

* Lazy build output eager tensors after job complete

* non scalar output test

* stale input

* add debug

* add

* add test

* add test

* deal with empty kernel of_record tensor buffer

* add test

* refine test

* New scope by stage id

* rm debug log

* finish test

* half impl of stage id infer

* Fix pipeline stage id by close max group

* remove note

* remove debug code

* Fea/checkpoint stop identity (#6216)

* Primitive (#6183)

* Add Primitive

* #ifdef WITH_CUDA

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Disable implicit boxing when parallel num eq one (#6188)

* mv_boxing_folder_to_core

* minor fix

* disable_implicit_boxing_when_parallel_num_eq_one

* Update eager_consistent_op_interpreter.cpp

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Lazy support Scalar (#6181)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Fix LayerNorm check bug (#6196)

* fix(Layernorm): fix check bug

* fix judge whether cpu or not

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add glu op (#6065)

* add glu op

* del glu_op export，align with torch

* mod glu_op

* mov op logic to C++

* Solve problems

* solve conflict

* delete gradient functor

* add ndim check

* add GLU test

* delete blank line

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>

* Primitive based copy task node (#6195)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* KernelState (#6198)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* container_util: fix VectorAt, remove useless MutMapAt (#6172)

* fcontainer_util: fix VectorAt, remove useless MutMapAt

* fcontainer_util: format

* MapAt: add default value version

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Refine StreamContext (#6191)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu symetric s to s (#6153)

* mv_boxing_folder_to_core

* minor fix

* cpu_symetric_s_to_s

* add test case

* auto format by CI

* minor fix

* refine

* Update eager_nccl_kernels.cpp

* minor fix

* fix bug

* minor fix

* Update oneflow/user/kernels/eager_nccl_kernels.cpp

Co-authored-by: daquexian <daquexian566@gmail.com>

* Update eager_nccl_kernels.cpp

* Update eager_nccl_kernels.cpp

* minor fix

* Update eager_nccl_kernels.cpp

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>

* fix bug (#6197)

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix consistent tensor zeros (#6202)

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* [Feat.] nn.Graph support grad acc with input/output tensor (#6155)

* nn.Graph support grad acc with input/output tensor

* dirty pass grad acc

* revert tensor.backward hack

* fix indent

* default S0 -> B

* Pack op/kernel support scalar input

* nn.Graph output pack support loss scalar

* add test script

* pass test

* Lazy build output eager tensors after job complete

* non scalar output test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Dev eliminate gcc warnings (#6199)

* fix gcc warning

* refine

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* StreamContextAdapter (#6205)

* StreamContextAdapter

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Autotest generate input tensor (#6206)

* Add tensor yaml, support export tensor functional api.

* refine

* Remove packed functor signature

* remove unused file

* Refine

* refine

* add activation op import

* reinit oneflow init.py

* add oneflow abs and exp

* add oneflow abs and exp

* add acos

* add arccosh

* add more op

* add more ops

* add more op

* add more ops

* add log1p

* add more smaples

* add more ops

* add more ops

* add more ops

* add more ops

* Complete tensor functional apis.

* Fix pybind call

* add more ops

* add ops done

* Add target of_functional_tensor_obj

* Disable throw visibility warnings

* fix target link

* fix

* fix incorrect use of flow.Tensor.

* Fix error merge

* fix

* fix add unittest

* refine

* refine

* fix

* fix

* add tensor doc

* auto format by CI

* refine

* Fix

* Add doc for python function

* refine

* add tensor method docstring

* fix some bug

* fix docs bug

* Fix

* auto format by CI

* Tensor->tensor

* Tensor->tensor

* refine Tensor->tensor

* fix

* fix

* fix

* fix conflict

* fix bug

* fix ci bug

* fix

* delete diag op

* fix conflict

* Fix segment

* fix

* merge

* autotest framework generate input tensor

* autotest framework generate input tensor

* fix bug

* fix impl bug

* refine

* refine

* refine

* fix

* fix

* fix comments

* delete useless

* fix ci error

* fix ci error

Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cleanup KernelUtil (#6212)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Rename flow to oneflow in user hint (#6190)

* style(*): rename flow to oneflow in user hint

* fix(*): fix doctest

* auto format by CI

* remove ddp speed test

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>

* merg and refactor

* refact code

* add io identity for activation checkpointing

Co-authored-by: Juncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: cheng cheng <472491134@qq.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: QiangX-man <87475073+QiangX-man@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: ZZK <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>

* rm useless code

* refien test

* add skip

* Fix bug of Multi-Client src tick output order

* refactor

* test pass

* waning to info

* import os

* fix test

* auto format by CI

* refactor fix pipeline stage id pass algo

* add refine test

* insert identity when do pipeline

* refine test

* fix copy buffer

* refien pipeline demo

* format

* Add input/output ctrl edge to DstSubTick for order io and callback_notify

* add test scripts

* remove note

* add comment for using pipeling

* add comment

* rm useless to

* avoid cpu run cuda test

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: peitingkuan <peitingkuan@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Juncheng <liujuncheng1022@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: QiangX-man <87475073+QiangX-man@users.noreply.github.com>
Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: ZZK <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lixinqi lixinqi lixinqi approved these changes

Assignees
No one assigned
Labels
automerge enhancement system
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@liujuncheng @lixinqi @guo-ran @oneflow-ci-bot