[Dynamic] Check whether there is any inplace operation affecting gradient calculation

[liym27]

PR types

New features

PR changes

Others

Describe

The next PR of #27471
Deal with the backward to support assignment to a Variable in dynamic mode.

---

What this PR do ?

1. Add a new attribute version for VarBase

version is used to check whether an inplace operation will result in an incorrect gradient calculation.
The version number is incremented whenever the current VarBase is modified through an inplace operation.

import paddle
var = paddle.ones(shape=[4, 2, 3], dtype="float32")
print(var.inplace_version)  # 0

var[1] = 2.2
print(var.inplace_version)  # 1

2. Raise error if an inplace operation can result in incorrect gradient computation.

• What kind in-place operations will raise error in backward?
  1. The gradient computation uses the value of a Tensor and this Tensor is modified inplace after using it
     For example:

import paddle
var_a = paddle.ones(shape=[4, 2, 3], dtype="float32")
var_a.stop_gradient = False

var_b = var_a**2

# Here, the gradient computation will use the value of var_b
var_c = var_b**2
var_b[1:2] = 3.3  # var_b is modified inplace after using it

loss = paddle.nn.functional.relu(var_c)
loss.backward()

----------------------
Error Message Summary:
----------------------
PermissionDeniedError: Tensor 'eager_tmp_2' used in gradient computation in grad op 'elementwise_pow_grad' has been modified by an inplace operation. Its version is 1 but the expected version is 0. Please fix your code to void calling an inplace operator after using the Tensor.
  [Hint: Expected variable_version == wrapper_version, but received variable_version:1 != wrapper_version:0.] (at /paddle/Paddle_0722/Paddle/paddle/fluid/imperative/basic_engine.cc:229)

• What kind in-place operations will NOT raise error in backward?
  1. The gradient computation does not use the value of a Tensor through this Tensor is modified inplace
  2. The gradient computation uses the value of a Tensor but this Tensor is modified inplace before using it

Example for i

var_a = paddle.ones(shape=[4, 2, 3], dtype="float32")
var_a.stop_gradient = False

var_b = var_a**2
var_b[1:2] = 3  # var_b is modified inplace before using it
var_c = var_b + var_b  # Here, the grad op of sum doesn't use the value of var_b
loss = var_c.sum()

var_b[1:2] = 3  # var_b is modified inplace after using it

loss.backward()

Example for ii

var_a = paddle.ones(shape=[4, 2, 3], dtype="float32")
var_a.stop_gradient = False

var_b = var_a**2
var_b[1:2] = 3  # var_b is modified inplace before using it

# Here, the gradient computation will use the value of var_b
var_c = var_b**2
loss = var_c.sum()
loss.backward()

3. Add a new interface _bump_inplace_version() for VarBase to bump the version whenever the Tensor is modified through an inplace operation.

4. For api assign, call _bump_inplace_version() when it's an inplace operation inn dynamic mode

---

Speed comparison

• seq2seq :
  batch num: 20*100
  batch size: 32

• resnet:
  batch num: 10*100
  batch size: 2

• ptb：
  batch num: 200*100
  batch size:100

Modle	without inplace (s/batch)	with inplace (s/batch)
cpu seq2se	0.0650103	0.0661232	- 1.712%
cpu resnet	2.53313	2.54028	-0.282%
cpu ptb	0.0550649	0.0555787	-0.925%
gpu ptb	0.018393	0.0187718	-2.0179%

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[zhhsplendid]

LGTM

[luotao1]

LGTM for tensor.h! 注释写的太太太棒了！

[zhhsplendid]

LGTM

[luotao1]

LGTM