Add 12 inplace APIs including auto generated #32573

pangyoki · 2021-04-26T08:05:25Z

PR types

New features

PR changes

APIs

Describe

Inplace Strategy has been implemented in PR #30103 .
Add 12 Inplace APIs in this PR.

NOTICE

The newly added inplace APIs in this PR will not be used publicly. They can only be called in the form of Tensor.api_, but CANNOT be called in the form of paddle.api_.

There is no corresponding Tensor.api method for activation op, such as relu, hard_sigmoid, leaky_relu, sigmoid and , so its inplace method will not be added temporarily.
The input of add_n API is a Tensor list, and it is not convenient to use Tensor.api to call, so its inplace method will not be added temporarily.

Processed API Lists in this PR

Inplace OP	paddle下API
clip	paddle.tensor.math.clip
scale	paddle.tensor.math.scale
elementwise_sub	paddle.tensor.math.subtract
elementwise_add	paddle.tensor.math.add

Add 8 Inplace APIs that are auto generated.

Inplace OP	paddle下API
ceil	paddle.tensor.math.ceil
floor	paddle.tensor.math.floor
exp	paddle.tensor.math.exp
reciprocal	paddle.tensor.math.reciprocal
round	paddle.tensor.math.round
sqrt	paddle.tensor.math.sqrt
rsqrt	paddle.tensor.math.rsqrt

PS: square grad op needs to use the input var, so the corresponding Inplace API square_ hasn't been added.

Fix flatten_contiguous_range op kernel, and add flatten_ Inplace API.

Inplace OP	paddle下API
flatten_contiguous_range	paddle.tensor.manipulation.flatten

17 Inplace APIs in Tensor method

Tensor.reshape_
Tensor.squeeze_
Tensor.unsqueeze_
Tensor.scatter_
Tensor.tanh_
Tensor.clip_
Tensor.scale_
Tensor.add_
Tensor.subtract_
Tensor.ceil_
Tensor.floor_
Tensor.exp_
Tensor.reciprocal_
Tensor.round_
Tensor.sqrt_
Tensor.rsqrt_
Tensor.flatten_

example to use Inplace APIs

import paddle

a = paddle.rand([2,3])
# [[0.45528218, 0.92556411, 0.90491837],
#   [0.31372914, 0.01990727, 0.94381261]]

b = paddle.ones([2,3])
# [[1., 1., 1.],
# [1., 1., 1.]]

a.add_(b)
print(a)
# [[1.45528221, 1.92556405, 1.90491843],
#  [1.31372917, 1.01990724, 1.94381261]]

paddle.add_(a,b)
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# AttributeError: module 'paddle' has no attribute 'add_'

paddle-bot-old · 2021-04-26T08:05:28Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… add_18_inplace_apis

zhiqiu

LGTM

jzhang533

LGTM for tools/wlist.json now, but we do have plan to gradually cleanup this white list.

jzhang533 · 2021-04-29T09:11:22Z

python/paddle/nn/functional/loss.py

+
+
+@inplace_apis_in_dygraph_only
+def softmax_with_cross_entropy_(logits,


softmax_with_cross_entropy API 已经被标记为deprecated了，所以这个inplace版本应该是也不需要了。

已经删除了inplace版本的softmax_with_cross_entropy_，谢谢

zhiqiu

LGTM

XiaoguangHu01

LGTM

jzhang533

LGTM

* add relu6_ hardsigmoid_ leaky_relu_ Inplace APIs * add softmax_with_cross_entropy_ Inplace API * add clip_ scale_ add_ subtract_ Inplace APIs * add wlist * fix parameter of scale api * add add_n_ Inplace API and remove log_ Inplace API * fix elementwise_add_ and elementwise_sub_ broadcast problem * elementwise inplace api give error message before run the op * use broadcast_shape in elementwise inplace op * add 8 inplace apis that is auto generated * add unittest for all inplace apis * add decorator for inplace apis in static mode * fix windows blas fail of exp inplace api, change array_equal to allclose * add flatten inplace api * add flatten unittest * fix flatten unittest * add decorator * fix grad.numpy in test_pylayer_op * unsupport softmax_with_cross_entropy_ * add test_inplace_softmax_with_cross_entropy to static_mode_white_list * delete __all__ in inplace_utils * delete activation inplace function and add Tensor.inplace_func * change paddle.inplace_ to Tensor.inplace_ * fix little problem * add paddle in inplace_utils

pangyoki added 19 commits January 20, 2021 09:10

add relu6_ hardsigmoid_ leaky_relu_ Inplace APIs

19b4e1c

add softmax_with_cross_entropy_ Inplace API

cd325a4

add clip_ scale_ add_ subtract_ Inplace APIs

7b71976

add wlist

8a3878c

fix parameter of scale api

71db93a

add add_n_ Inplace API and remove log_ Inplace API

ee118ec

fix elementwise_add_ and elementwise_sub_ broadcast problem

188960f

elementwise inplace api give error message before run the op

0698f48

use broadcast_shape in elementwise inplace op

a2cec25

add 8 inplace apis that is auto generated

ec71854

add unittest for all inplace apis

234f7d8

add decorator for inplace apis in static mode

6c20097

fix windows blas fail of exp inplace api, change array_equal to allclose

46ba29c

add flatten inplace api

ae14bbe

add flatten unittest

ea0777a

fix flatten unittest

40ccfe9

merge 18 inplace apis

52190d5

add decorator

af6a388

solve conflict

dc4ad2c

pangyoki added 4 commits April 26, 2021 08:07

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

43fc847

… add_18_inplace_apis

fix grad.numpy in test_pylayer_op

d330959

solve conflict of __init__

5a357c0

solve conflict of __init__ v2

388afb1

zhiqiu previously approved these changes Apr 29, 2021

View reviewed changes

raindrops2sea requested review from XiaoguangHu01 and phlrain April 29, 2021 07:24

lanxianghit previously approved these changes Apr 29, 2021

View reviewed changes

jzhang533 reviewed Apr 29, 2021

View reviewed changes

unsupport softmax_with_cross_entropy_

29737d4

pangyoki dismissed stale reviews from lanxianghit and zhiqiu via 29737d4 April 29, 2021 09:37

add test_inplace_softmax_with_cross_entropy to static_mode_white_list

bc90468

pangyoki changed the title ~~Add 18 inplace APIs including auto generated~~ Add 17 inplace APIs including auto generated Apr 29, 2021

pangyoki added 4 commits April 29, 2021 09:54

delete __all__ in inplace_utils

a5293e6

solve conflict

94b2713

delete activation inplace function and add Tensor.inplace_func

eb11b3b

change paddle.inplace_ to Tensor.inplace_

1001d9e

pangyoki changed the title ~~Add 17 inplace APIs including auto generated~~ Add 12 inplace APIs including auto generated Apr 29, 2021

pangyoki added 2 commits April 29, 2021 18:27

fix little problem

4faff99

add paddle in inplace_utils

f46d3fe

zhiqiu approved these changes Apr 30, 2021

View reviewed changes

XiaoguangHu01 approved these changes Apr 30, 2021

View reviewed changes

jzhang533 approved these changes Apr 30, 2021

View reviewed changes

pangyoki merged commit 308073d into PaddlePaddle:develop Apr 30, 2021

pangyoki mentioned this pull request Apr 30, 2021

【Cherry pick PR 32573】Add 12 inplace APIs including auto generated #32699

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add 12 inplace APIs including auto generated #32573

Add 12 inplace APIs including auto generated #32573

Uh oh!

pangyoki commented Apr 26, 2021 •

edited

Loading

Uh oh!

paddle-bot-old bot commented Apr 26, 2021

Uh oh!

zhiqiu left a comment

Uh oh!

jzhang533 left a comment

Uh oh!

jzhang533 Apr 29, 2021

Uh oh!

pangyoki Apr 29, 2021

Uh oh!

zhiqiu left a comment

Uh oh!

XiaoguangHu01 left a comment

Uh oh!

jzhang533 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants



		@inplace_apis_in_dygraph_only
		def softmax_with_cross_entropy_(logits,

Uh oh!

Add 12 inplace APIs including auto generated #32573

Add 12 inplace APIs including auto generated #32573

Uh oh!

Conversation

pangyoki commented Apr 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR types

PR changes

Describe

NOTICE

Processed API Lists in this PR

17 Inplace APIs in Tensor method

example to use Inplace APIs

Uh oh!

paddle-bot-old bot commented Apr 26, 2021

Uh oh!

zhiqiu left a comment

Choose a reason for hiding this comment

Uh oh!

jzhang533 left a comment

Choose a reason for hiding this comment

Uh oh!

jzhang533 Apr 29, 2021

Choose a reason for hiding this comment

Uh oh!

pangyoki Apr 29, 2021

Choose a reason for hiding this comment

Uh oh!

zhiqiu left a comment

Choose a reason for hiding this comment

Uh oh!

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

Uh oh!

jzhang533 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pangyoki commented Apr 26, 2021 •

edited

Loading