[Precision Depth Alignment] paddle.log_sigmoid

[zhengshengning]

PR Category

Operator Mechanism

PR Types

Performance

Description

torch实现 ： aten/src/ATen/native/cuda/ActivationLogSigmoidKernel.cu

void launch_log_sigmoid_forward_kernel(TensorIteratorBase& iter) {
  AT_DISPATCH_FLOATING_TYPES_AND2(
      kHalf, kBFloat16, iter.common_dtype(), "log_sigmoid_forward_cuda", [&] {
        using opmath_t = at::opmath_type<scalar_t>;

        gpu_kernel(iter, [] GPU_LAMBDA(scalar_t in_) -> scalar_t {
          const opmath_t in = in_;
          const auto min = std::min(opmath_t(0), in);
          const auto z = std::exp(-std::abs(in));
          return min - std::log1p(z);
        });
      });
}

主要修改：

1. 前向计算 (CudaLogSigmoidFunctor)
   旧实现： - (max(-x, 0) + log(exp(-max(-x, 0)) + exp(-x - max(-x, 0)))) ；
   新实现： log_sigmoid(x) = min(0,x) - log1p(exp(-|x|)) ；
2. 反向计算 (CudaLogSigmoidGradFunctor)
   旧实现： dx = dout * exp(-x - max(-x, 0)) / (exp(-max(-x, 0)) + exp(-x - max(-x,0))) ；
   新实现：grad = dout * (max_deriv - sign * (z / (1 + z))) ; z = exp(-abs(x)); max_deriv = (x < 0) ? 1 : 0, sign = (x < 0) ? 1 : -1 ;

测试结果：
paddle.nn.functional.log_sigmoid 对齐case（共16 个）

pcard-93269

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[Eddie-Wang1120]

LGTM

[A-nnonymous]

LGTM