refine tensor doc and add module to_global doc #7823

hjchen2 · 2022-03-17T04:21:54Z

完善Tensor.to_global接口文档，增加对grad_sbp的描述

完善Tensor.to_local接口文档

增加Tensor Attributes文档

增加Module.to_consistent和Module.to_global接口的文档

python/oneflow/framework/docstr/module.py

python/oneflow/framework/docstr/tensor.py

chengtbf · 2022-03-17T06:36:37Z

python/oneflow/framework/docstr/global_cast.py

-    Cast a local tensor to global tensor or cast a
-    global tensor to another global tensor with
-    different sbp or placement
+    Convert a local tensor to global tensor or convert a global tensor to another global tensor with


拆成两段

chengtbf · 2022-03-17T06:37:43Z

python/oneflow/framework/docstr/global_cast.py

-        sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional): the desired sbp descriptor of returned global tensor. Default: if None, the input tensor must be consistent one and use its own sbp.
+        placement (flow.placement, optional): the desired placement of returned global tensor. Default: if None, the input tensor must be global and use its own placement.
+        sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional): the desired sbp descriptor of returned global tensor. Default: if None, the input tensor must be global and use its own sbp. 
+        grad_sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional): manually specify the gradient sbp of the operation in the backward pass. Default: if None, the gradient sbp will be infered automatically.


Example 需要至少增加一个例子。

local_tensor -> global tensor S0 tensor shape

从 release note 借鉴过来

S0 的约束；
B 的约束； Note: B 会发生 rank 0 的数据覆盖

使用 api doc 的 NOTE 功能描述上述的重点约束

更新了，再review一下

Co-authored-by: Yao Chi <later@usopp.net>

python/oneflow/framework/docstr/module.py

chengtbf · 2022-03-18T09:28:30Z

docs/source/tensor.rst

@@ -156,7 +156,8 @@ OneFlow Tensor Class
            tan, 
            tanh, 
            tile, 
-            to, 
+            to,
+            to_consistent,


这个是按照字典序排的吗，还是随机排的。 to_consistent 是过时接口，是不是放在 to_global 下面比较好。字典序排的就不用改了

挪到to_local下面去了

chengtbf · 2022-03-18T09:29:12Z

python/oneflow/framework/docstr/module.py

+
+
+    Note:
+        This method modifies the module in-place.


这里要分情况？如果原有的 module 都是 local tensor，这里就不是 inplace 的吧。local tensor -> global tensor

是的，local to global无法Inplace

这个是说原地修改module，不是改tensor

这个inplace针对的对象是 module，针对于module是改变自身的操作

chengtbf · 2022-03-18T09:29:46Z

python/oneflow/framework/docstr/module.py

+        >>> m = flow.nn.Conv2d(in_channels=3, out_channels=4, kernel_size=3)
+        >>> m.to_global(placement=flow.placement("cpu", ranks=[0]), sbp=[flow.sbp.split(0)])
+        >>> m.weight.is_global
+        True


加一个：

>>> m.bias.is_global True

chengtbf · 2022-03-18T09:40:41Z

总体上已经觉得写的很好了~ 值得推广

strint · 2022-03-18T10:13:19Z

python/oneflow/framework/docstr/module.py

+add_docstr(
+    oneflow.nn.Module.to_consistent,
+    """
+    This interface is no longer available, please use :func:`oneflow.nn.Module.to_global` instead


instead.

少个句号

strint · 2022-03-18T10:19:45Z

python/oneflow/framework/docstr/tensor.py

+    Args:
+        placement (flow.placement, optional): the desired placement of returned global tensor. Default: None
+        sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional): the desired sbp of returned global tensor. Default: None
+        grad_sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional): manually specify the gradient sbp of this operation in the backward pass. If None, the gradient sbp will be infered automatically. It is only used if this tensor is a global tensor. Default: None


specify the sbp of this tensor's grad tensor in the backward pass ?

strint · 2022-03-18T10:27:58Z

python/oneflow/framework/docstr/tensor.py

+
+        >>> # results on rank 0
+        oneflow.Size([2])
+        tensor([0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(axis=0),), dtype=oneflow.float32)


这里b to s，除了sbp，看起来什么都没变化。

感觉需要体现一下里面的local tensor变化了，也就是自动做了split？

这个在to_local 接口文档中有所体现，这里提及to_local是不是有点越界？

strint · 2022-03-18T10:42:29Z

python/oneflow/framework/docstr/tensor_attributes.py

+
+    ``oneflow.sbp`` includes three types:
+
+        - oneflow.sbp.split(axis)


发现split接口这里一个问题，torch下，基本都是叫dim，tf下才叫axis

https://pytorch.org/docs/stable/search.html?q=dim&check_keywords=yes&area=default#

https://stackoverflow.com/questions/62333053/whats-the-difference-between-dim-in-pytorch-and-axis-in-tensorflow

如果在考虑torch的语境的话，这里我们最好改为dim

lixinqi · 2022-03-18T10:58:21Z

python/oneflow/framework/docstr/tensor_attributes.py

+    A ``oneflow.sbp`` is an object representing that how the data of the global tensor is distributed across the ranks of the ``Tensor`` placement.
+
+    ``oneflow.sbp`` includes three types:


这里的表述有点奇怪，oneflow.sbp是一个python module，而oneflow.sbp.sbp是一个类。

A oneflow.sbp.sbp is a distribution descriptor object representing how the data of the global tensor is distributed across the ranks of the Tensor placement.

There are three types of distribution descriptor instances in module oneflow.sbp:

clackhan · 2022-03-18T12:20:26Z

增加Tensor Attributes文档(device)

完善Tensor.device接口文档

完善Tensor.placement接口文档

完善Tensor.sbp接口文档

完善Tensor.is_global接口文档

oneflow.to()接口

…-Inc/oneflow into dev_refine_to_global_doc

github-actions · 2022-03-18T16:02:37Z

CI failed when running job: cuda-module-distributed-rank-1. PR label automerge has been removed

github-actions · 2022-03-18T16:02:37Z

CI failed when running job: cuda-module-distributed-rank-0. PR label automerge has been removed

github-actions · 2022-03-18T17:25:01Z

Speed stats:

refine tensor doc and add module to_global doc

314fa69

hjchen2 requested review from doombeaker, jackalcooper, BBuf and daquexian as code owners March 17, 2022 04:21

doombeaker reviewed Mar 17, 2022

View reviewed changes

python/oneflow/framework/docstr/module.py Outdated Show resolved Hide resolved

python/oneflow/framework/docstr/module.py Outdated Show resolved Hide resolved

python/oneflow/framework/docstr/tensor.py Outdated Show resolved Hide resolved

chengtbf reviewed Mar 17, 2022

View reviewed changes

hjchen2 and others added 4 commits March 17, 2022 15:15

Update python/oneflow/framework/docstr/module.py

5339f86

Co-authored-by: Yao Chi <later@usopp.net>

Update python/oneflow/framework/docstr/module.py

d9083aa

Co-authored-by: Yao Chi <later@usopp.net>

Update python/oneflow/framework/docstr/tensor.py

b7c0560

Co-authored-by: Yao Chi <later@usopp.net>

refine global tensor doc

9262a3f

chengtbf reviewed Mar 18, 2022

View reviewed changes

chengtbf approved these changes Mar 18, 2022

View reviewed changes

strint reviewed Mar 18, 2022

View reviewed changes

strint approved these changes Mar 18, 2022

View reviewed changes

strint reviewed Mar 18, 2022

View reviewed changes

lixinqi reviewed Mar 18, 2022

View reviewed changes

clackhan and others added 2 commits March 18, 2022 20:23

add device doc

00adf1c

Merge branch 'master' into dev_refine_to_global_doc

a12e079

hjchen2 requested a review from oneflow-ci-bot March 18, 2022 12:23

Merge branch 'dev_refine_to_global_doc' of https://github.com/Oneflow…

1d17235

…-Inc/oneflow into dev_refine_to_global_doc

hjchen2 added automerge documentation python labels Mar 18, 2022

clackhan and others added 2 commits March 18, 2022 20:29

revert dim to axis

e996f1d

refine

3c80c07

hjchen2 and others added 4 commits March 18, 2022 20:53

Merge branch 'dev_refine_to_global_doc' of https://github.com/Oneflow…

1eb97e7

…-Inc/oneflow into dev_refine_to_global_doc

Merge branch 'master' into dev_refine_to_global_doc

6fa895c

fix

eb88c94

Merge branch 'dev_refine_to_global_doc' of https://github.com/Oneflow…

ac24673

…-Inc/oneflow into dev_refine_to_global_doc

hjchen2 enabled auto-merge (squash) March 18, 2022 15:38

github-actions bot removed the automerge label Mar 18, 2022

hjchen2 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 18, 2022 16:33

clackhan added 2 commits March 19, 2022 08:42

fix doc testbug

9a478cf

Merge branch 'master' into dev_refine_to_global_doc

028a85c

clackhan added the automerge label Mar 19, 2022

clackhan requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 19, 2022 00:43

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 19, 2022 00:46

hjchen2 merged commit 42fe221 into master Mar 19, 2022

hjchen2 deleted the dev_refine_to_global_doc branch March 19, 2022 03:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refine tensor doc and add module to_global doc #7823

refine tensor doc and add module to_global doc #7823

hjchen2 commented Mar 17, 2022 •

edited

Loading

chengtbf Mar 17, 2022

chengtbf Mar 17, 2022

chengtbf Mar 17, 2022

chengtbf Mar 17, 2022

hjchen2 Mar 18, 2022

chengtbf Mar 18, 2022

hjchen2 Mar 18, 2022

chengtbf Mar 18, 2022 •

edited

Loading

strint Mar 18, 2022

hjchen2 Mar 18, 2022

clackhan Mar 18, 2022

chengtbf Mar 18, 2022

chengtbf commented Mar 18, 2022

strint Mar 18, 2022

hjchen2 Mar 18, 2022

strint Mar 18, 2022

hjchen2 Mar 18, 2022

strint Mar 18, 2022

clackhan Mar 18, 2022

strint Mar 18, 2022

lixinqi Mar 18, 2022 •

edited

Loading

clackhan commented Mar 18, 2022

github-actions bot commented Mar 18, 2022

github-actions bot commented Mar 18, 2022

github-actions bot commented Mar 18, 2022


		``oneflow.sbp`` includes three types:

		- oneflow.sbp.split(axis)

		A ``oneflow.sbp`` is an object representing that how the data of the global tensor is distributed across the ranks of the ``Tensor`` placement.

		``oneflow.sbp`` includes three types:

refine tensor doc and add module to_global doc #7823

refine tensor doc and add module to_global doc #7823

Conversation

hjchen2 commented Mar 17, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chengtbf Mar 18, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chengtbf commented Mar 18, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lixinqi Mar 18, 2022 • edited Loading

Choose a reason for hiding this comment

clackhan commented Mar 18, 2022

github-actions bot commented Mar 18, 2022

github-actions bot commented Mar 18, 2022

github-actions bot commented Mar 18, 2022

hjchen2 commented Mar 17, 2022 •

edited

Loading

chengtbf Mar 18, 2022 •

edited

Loading

lixinqi Mar 18, 2022 •

edited

Loading