add index_select op #5661

Ikkyu321 · 2021-07-29T09:38:34Z

CLAassistant · 2021-07-29T09:38:40Z

All committers have signed the CLA.

python/oneflow/nn/modules/index_select.py

python/oneflow/test/modules/test_index_select.py

python/oneflow/nn/modules/index_select.py

python/oneflow/test/modules/test_index_select.py

…flow into dev_index_select git pull project needed

hengzi · 2021-08-05T06:35:32Z

添加文档截图参考 #5636

oneflow/docs 执行 make html 生成html文件，下载到电脑用浏览器打开

python/oneflow/nn/modules/index_select.py

doombeaker · 2021-08-06T07:30:13Z

python/oneflow/nn/modules/index_select.py

+
+
+@register_tensor_op("index_select")
+def index_select_op(input, dim, index, sparse_grad=False):


flow.xxx 和 flow.Tensor.xxx 共用一个函数的话，就共用了 docstring，那么总有一方的文档是不准确的。
需要为 flow.Tensor.xxx 单独准备一个函数，书写 docstring。
可以参考 acos 等算子的写法

flow.xxx 和 flow.Tensor.xxx 共用一个函数的话，就共用了 docstring，那么总有一方的文档是不准确的。
需要为 flow.Tensor.xxx 单独准备一个函数，书写 docstring。
可以参考 acos 等算子的写法

已修改

按照 torch 的 API 文档 https://pytorch.org/docs/stable/generated/torch.index_select.html?highlight=index_select#torch.index_select

这个算子的原型是：

torch.index_select(input, dim, index, *, out=None) → Tensor

考虑到暂时不对齐 out 参数，那么原型也应该是

.index_select(input, dim, index)

这里的 sparse_grad 是那个版本的呢？

PR 描述里的截图是不对的，应该更行

flow.Tensor.index_select 的截图也应该补上

doombeaker · 2021-08-06T13:29:28Z

python/oneflow/nn/modules/index_select.py

+    input.index_select(dim, index, sparse_grad) -> Tensor
+    See :func:`oneflow.index_select`
+    """
+    assert sparse_grad is False, "Only support bool = False for now!"


不必要把一样的代码写两遍，直接调用 index_select_op 就可以了

index_select_op(input, dim, index, sparse_grad)

doombeaker · 2021-08-06T13:32:07Z

python/oneflow/nn/modules/index_select.py

+
+
+@register_tensor_op("index_select")
+def index_select_op(input, dim, index, sparse_grad=False):


按照 torch 的 API 文档 https://pytorch.org/docs/stable/generated/torch.index_select.html?highlight=index_select#torch.index_select

这个算子的原型是：

torch.index_select(input, dim, index, *, out=None) → Tensor

考虑到暂时不对齐 out 参数，那么原型也应该是

.index_select(input, dim, index)

这里的 sparse_grad 是那个版本的呢？

doombeaker · 2021-08-06T13:33:11Z

python/oneflow/nn/modules/index_select.py

+
+
+@register_tensor_op("index_select")
+def index_select_op(input, dim, index, sparse_grad=False):


PR 描述里的截图是不对的，应该更行

flow.Tensor.index_select 的截图也应该补上

doombeaker · 2021-08-06T13:34:22Z

python/oneflow/nn/modules/index_select.py

+    Args:
+        input (Tensor): the source tensor
+        dim (int): the axis along which to index
+        index (LongTensor): the indices of elements to select


flow 里面还没有 LongTensor

the 1-D tensor containing the indices to index

我给的建议是直接参考 torch 的

the 1-D tensor containing the indices to index

我给的建议是直接参考 torch 的

已修改

doombeaker · 2021-08-06T13:34:54Z

python/oneflow/nn/modules/index_select.py

+
+
+def index_select_op(input, dim, index, sparse_grad=False):
+    r"""Select values along an axis specified by `dim`.


加一个对 PyTorch 参考的说明。可以参考 Conv1d

加一个对 PyTorch 参考的说明。可以参考 Conv1d

已修改

doombeaker · 2021-08-06T13:43:33Z

python/oneflow/nn/modules/index_select.py

+    """
+    assert sparse_grad is False, "Only support bool = False for now!"
+    assert len(index.shape) == 1, "Dimensions of index should be an LongTensor"
+    assert dim < len(input.shape) and dim > -1, "Value of dim is out of range"


Suggested change

assert dim < len(input.shape) and dim > -1, "Value of dim is out of range"

assert dim < len(input.shape) and dim >= 0, "Value of dim is out of range"

doombeaker · 2021-08-06T13:44:53Z

python/oneflow/nn/modules/index_select.py

+    assert sparse_grad is False, "Only support bool = False for now!"
+    assert len(index.shape) == 1, "Dimensions of index should be an LongTensor"
+    assert dim < len(input.shape) and dim > -1, "Value of dim is out of range"
+    assert _input_args_is_int(index.tolist()), "input index parameter is not illegal!"


我觉得这里的提示可以更明确点，告诉用户怎样做可以修正好。而不是只告诉用户他做的不合法

我觉得这里的提示可以更明确点，告诉用户怎样做可以修正好。而不是只告诉用户他做的不合法

已修改

doombeaker · 2021-08-06T13:49:09Z

python/oneflow/nn/modules/index_select.py

+@register_tensor_op("index_select")
+def index_select_op_tensor(input, dim, index, sparse_grad=False):
+    """
+    input.index_select(dim, index, sparse_grad) -> Tensor


函数原型是错的（不应该有 sparase_grad）

可以直接参考 https://pytorch.org/docs/1.9.0/generated/torch.Tensor.index_select.html?highlight=index_select#torch.Tensor.index_select 或者 oneflow 已有的其它例子

函数原型是错的（不应该有 sparase_grad）

可以直接参考 https://pytorch.org/docs/1.9.0/generated/torch.Tensor.index_select.html?highlight=index_select#torch.Tensor.index_select 或者 oneflow 已有的其它例子

已修改

doombeaker · 2021-08-06T13:50:41Z

python/oneflow/test/modules/test_index_select.py

+        device = random_device()
+
+        # test 4 dimensions tensor
+        axis = random(0, 4).to(int)


我不是很赞同 axis 和 dim 混用。虽然无伤大雅

我不是很赞同 axis 和 dim 混用。虽然无伤大雅

修改为dim

doombeaker · 2021-08-06T13:51:00Z

python/oneflow/nn/modules/index_select.py

+            x = index_i.expand(index_rshp)
+            index_gather = flow.cat((index_gather, x), dim)
+
+    return flow.gather(input, index_gather, dim, sparse_grad)


Suggested change

return flow.gather(input, index_gather, dim, sparse_grad)

return flow.gather(input, index_gather, dim)

doombeaker · 2021-08-12T05:23:09Z

python/oneflow/test/modules/test_index_select.py

+            tensor_dim.append(random(2, 6).to(int).value())
+
+        index = random_pytorch_tensor(
+            ndim=1, low=0, high=tensor_dim[dim.value()], dtype=int


Suggested change

ndim=1, low=0, high=tensor_dim[dim.value()], dtype=int

ndim=1, dim0=dim.value()+1, low=0, high=tensor_dim[dim.value()], dtype=int

现在这样的测试案例，index 是shape 总是 [1]，不能充分覆盖各种情况。

现在这样的测试案例，index 是shape 总是 [1]，不能充分覆盖各种情况。

已修改index的长度为随机（1,10）

…flow into dev_index_select

github-actions · 2021-08-12T21:38:15Z

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.8ms (= 7039.8ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.1ms (= 6402.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 140.8ms / 128.1ms)

PyTorch resnet50 time: 87.1ms (= 4353.6ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.5ms (= 3724.0ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.17 (= 87.1ms / 74.5ms)

PyTorch resnet50 time: 59.9ms (= 2996.0ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 48.5ms (= 2423.7ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.24 (= 59.9ms / 48.5ms)

PyTorch resnet50 time: 48.7ms (= 2435.6ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 38.1ms (= 1903.1ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.28 (= 48.7ms / 38.1ms)

PyTorch resnet50 time: 40.1ms (= 2004.1ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 36.2ms (= 1810.5ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.11 (= 40.1ms / 36.2ms)

add index_select op

34cfc1b

add CI testCase, update index_select op

8c71398

Ikkyu321 requested a review from hengzi August 2, 2021 09:34

Ikkyu321 added enhancement interface op labels Aug 2, 2021

Ikkyu321 marked this pull request as ready for review August 2, 2021 09:36

hengzi mentioned this pull request Aug 3, 2021

OneFlow 算子对齐 PyTorch 完备计划推进表 #4936

Closed

hengzi reviewed Aug 3, 2021

View reviewed changes

solve some problems of code reviewing

a98b86f

Ikkyu321 requested a review from hengzi August 3, 2021 07:23

hengzi reviewed Aug 3, 2021

View reviewed changes

python/oneflow/nn/modules/index_select.py Outdated Show resolved Hide resolved

python/oneflow/test/modules/test_index_select.py Show resolved Hide resolved

python/oneflow/test/modules/test_index_select.py Outdated Show resolved Hide resolved

hengzi and others added 4 commits August 3, 2021 17:12

Merge branch 'master' into dev_index_select

bef9232

update CI test

aedd7cb

Merge branch 'dev_index_select' of https://github.com/Oneflow-Inc/one…

4181a69

…flow into dev_index_select git pull project needed

update index_select, add oundary conditions

a235680

hengzi reviewed Aug 5, 2021

View reviewed changes

python/oneflow/nn/modules/index_select.py Outdated Show resolved Hide resolved

Ikkyu321 and others added 4 commits August 5, 2021 16:13

create html, changed code format

e516a10

Merge branch 'master' into dev_index_select

74f1966

Merge branch 'master' into dev_index_select

2126733

Merge branch 'master' into dev_index_select

76a7bc9

doombeaker reviewed Aug 6, 2021

View reviewed changes

python/oneflow/nn/modules/index_select.py Outdated Show resolved Hide resolved

doombeaker reviewed Aug 6, 2021

View reviewed changes

Ikkyu321 added 2 commits August 6, 2021 19:01

delete module, change for using function call

d8c2a94

add annotation

39435d7

doombeaker reviewed Aug 6, 2021

View reviewed changes

doombeaker and others added 2 commits August 6, 2021 08:52

Merge branch 'master' into dev_index_select

2904098

solve several problems of reviewing code

926060a

Ikkyu321 added 2 commits August 11, 2021 16:39

Merge branch 'master' into dev_index_select

22e3b81

Merge branch 'master' into dev_index_select

f3887f3

doombeaker reviewed Aug 12, 2021

View reviewed changes

doombeaker approved these changes Aug 12, 2021

View reviewed changes

Ikkyu321 and others added 3 commits August 12, 2021 14:06

modified tensor index to a longer tensor

2aac233

Merge branch 'dev_index_select' of https://github.com/Oneflow-Inc/one…

e7e65c4

…flow into dev_index_select

Merge branch 'master' into dev_index_select

a227c65

hengzi approved these changes Aug 12, 2021

View reviewed changes

hengzi added the automerge label Aug 12, 2021

Merge branch 'master' into dev_index_select

a75e1ce

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 12, 2021 08:36

Merge branch 'master' into dev_index_select

e7a4881

oneflow-ci-bot self-requested a review August 12, 2021 10:30

Merge branch 'master' into dev_index_select

90584a5

oneflow-ci-bot removed their request for review August 12, 2021 12:24

Merge branch 'master' into dev_index_select

3e534f5

oneflow-ci-bot self-requested a review August 12, 2021 13:51

Merge branch 'master' into dev_index_select

b4d1517

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 12, 2021 15:39

Merge branch 'master' into dev_index_select

5abd478

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 12, 2021 17:47

Merge branch 'master' into dev_index_select

c45b7c1

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 12, 2021 19:53

oneflow-ci-bot merged commit bcd3116 into master Aug 12, 2021

oneflow-ci-bot deleted the dev_index_select branch August 12, 2021 21:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add index_select op #5661

add index_select op #5661

Ikkyu321 commented Jul 29, 2021 •

edited

Loading

CLAassistant commented Jul 29, 2021 •

edited

Loading

hengzi commented Aug 5, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 6, 2021

doombeaker Aug 6, 2021

doombeaker Aug 6, 2021

doombeaker Aug 6, 2021

doombeaker Aug 6, 2021

doombeaker Aug 6, 2021

doombeaker Aug 6, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 9, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 9, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 9, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 9, 2021

doombeaker Aug 6, 2021 •

edited

Loading

Ikkyu321 Aug 9, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 9, 2021

doombeaker Aug 6, 2021

Ikkyu321 Aug 9, 2021

doombeaker Aug 12, 2021

Ikkyu321 Aug 12, 2021

github-actions bot commented Aug 12, 2021



		@register_tensor_op("index_select")
		def index_select_op(input, dim, index, sparse_grad=False):



		def index_select_op(input, dim, index, sparse_grad=False):
		r"""Select values along an axis specified by `dim`.

	assert dim < len(input.shape) and dim > -1, "Value of dim is out of range"
	assert dim < len(input.shape) and dim >= 0, "Value of dim is out of range"

	return flow.gather(input, index_gather, dim, sparse_grad)
	return flow.gather(input, index_gather, dim)

	ndim=1, low=0, high=tensor_dim[dim.value()], dtype=int
	ndim=1, dim0=dim.value()+1, low=0, high=tensor_dim[dim.value()], dtype=int

add index_select op #5661

add index_select op #5661

Conversation

Ikkyu321 commented Jul 29, 2021 • edited Loading

CLAassistant commented Jul 29, 2021 • edited Loading

hengzi commented Aug 5, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

doombeaker Aug 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 12, 2021

Ikkyu321 commented Jul 29, 2021 •

edited

Loading

CLAassistant commented Jul 29, 2021 •

edited

Loading

doombeaker Aug 6, 2021 •

edited

Loading