add acos_double_grad in dygraph composite

[zhiminzhang0830]

PR Category

Operator Mechanism

PR Types

New features

Description

Pcard-75624
添加acos的动态图二阶反向组合

精度对齐时，参考acos测试代码，使用[-0.95, 0.95]范围内数据作为输入。

与torch对齐验证代码：

import paddle
import numpy as np
import torch

np.random.seed(42)

x_np = np.random.uniform(-0.95, 0.95, size=(100, 10000)).astype("float32")
dout_np = np.random.rand(*x_np.shape).astype("float32")
dout_np2 = np.random.rand(*x_np.shape).astype("float32")


# paddle
x = paddle.to_tensor(x_np)
x.stop_gradient = False

dout_pd = paddle.to_tensor(dout_np)
dout_pd.stop_gradient = False

dout_pd2 = paddle.to_tensor(dout_np2)
dout_pd2.stop_gradient = False

y_pd = paddle.acos(x)

y_grad_pd = paddle.grad(y_pd, x, dout_pd, create_graph=True, retain_graph=True)[0]
y_grad_grad_pd = paddle.grad(y_grad_pd, x, dout_pd2, create_graph=True, retain_graph=True)[0]


# torch
x = torch.tensor(x_np, requires_grad=True)
y_pt = torch.acos(x)

dout_pt = torch.as_tensor(dout_np)
dout_pt.requires_grad = True

dout_pt2 = torch.as_tensor(dout_np2)
dout_pt2.requires_grad = True

y_grad_pt = torch.autograd.grad(y_pt, x, dout_pt, create_graph=True, retain_graph=True)[0]
y_grad_grad_pt = torch.autograd.grad(y_grad_pt, x, dout_pt2, create_graph=True, retain_graph=True)[0]


np.testing.assert_allclose(y_grad_pd.numpy(), y_grad_pt.detach().numpy(), rtol=1e-6)
print('1st order grad check passed')
np.testing.assert_allclose(y_grad_grad_pd.numpy(), y_grad_grad_pt.detach().numpy(), rtol=1e-6)
print('2nd order grad check passed')

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[CLAassistant]

All committers have signed the CLA.

[HydrogenSulfate]

参考:

Paddle/paddle/fluid/prim/api/composite_backward/composite_double_backward_api.h

Lines 625 to 645 in b3afa7b

	template <typename T>
	void silu_double_grad(const Tensor& x,
	const Tensor& out,
	const Tensor& out_grad,
	const Tensor& grad_x_grad,
	Tensor* grad_x,
	Tensor* grad_out_grad) {
	auto s = sigmoid<T>(x);
	auto tmp1 = scale<T>(s, -1.0, 1.0);
	auto tmp2 = scale<T>(tmp1 * x, 1.0, 1.0);
	auto grad_x_grad_mul_sigmoid = grad_x_grad * s;
	if (grad_out_grad) {
	auto ddout = grad_x_grad_mul_sigmoid * tmp2;
	set_output<T>(ddout, grad_out_grad);
	}
	if (grad_x) {
	auto dx = grad_x_grad_mul_sigmoid * out_grad *
	(scale<T>(tmp2 - out, 1.0, 1.0)) * tmp1;
	set_output<T>(dx, grad_x);
	}
	}

，把两个分支计算的公共部分提取出来（如1-x*x)，这样能避免重复的kernel调用和计算开销

[zhiminzhang0830]

已修改

[HydrogenSulfate]

另外PR描述里的验证代码，构造的张量形状太小，可以在百万级别的矩阵上验证一下。

[HydrogenSulfate]

Suggested change

	// acos grad grad : ddout = -((1-x*x)^(-0.5)) * ddx, dx = dy *
	// (-x)*((1-x*x)^(-1.5)) * ddx
	// ddout = -((1-x*x)^(-0.5)) * ddx
	// dx = dy * (-x)*((1-x*x)^(-1.5)) * ddx

[zhiminzhang0830]

已修改

[zyfncg]

第二个x最好写成grad_out，以防后面静态图使用出现一些奇怪的问题

[zhiminzhang0830]

已修改，辛苦再看下