[Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double #75426

zhengshengning · 2025-09-21T13:18:25Z

PR Category

Operator Mechanism

PR Types

New features

Description

PR #75363 中遗留了doule类型未能与torch对齐。
本Pr主要将beta和threshold两个参数的类型从float改为double，这样就可以完全与torch的精度对齐。
修改记录文档：https://ku.baidu-int.com/knowledge/HFVrC7hq1Q/pKzJfZczuc/RqSZ7wzMmI/PYmTSAOJppi_li

注：
因为既然发现了float精度丢失问题，本PR觉得将问题一次性改透，因此同时修改了fused_softplus和OneDNN的fused_softplus。

pcard-93269

paddle-bot · 2025-09-21T13:18:32Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… accuracy_stable_softplus_v2

codecov-commenter · 2025-10-11T10:07:10Z

Codecov Report

❌ Patch coverage is 91.13924% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@d4c0773). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
paddle/phi/kernels/onednn/softplus_kernel.cc	0.00%	3 Missing ⚠️
...addle/fluid/ir_adaptor/translator/op_translator.cc	92.85%	2 Missing ⚠️
paddle/phi/kernels/funcs/activation_functor.h	50.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop   #75426   +/-   ##
==========================================
  Coverage           ?   91.13%           
==========================================
  Files              ?       11           
  Lines              ?       79           
  Branches           ?        0           
==========================================
  Hits               ?       72           
  Misses             ?        7           
  Partials           ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

changeyoung98

LGTM

wanghuancoder

LGTM

wanghuancoder

给争胜点赞，Paddle早期很多算子使用了float作为参数类型，导致精度丢失。这个PR具有示范意义。也说明float改double牵涉面很广：

Save/Load兼容性
推理Pass兼容性，以及常规算子与Fused算子一致性
老算子库兼容性

zyfncg · 2025-10-13T11:04:26Z

paddle/phi/ops/yaml/legacy/static_ops.yaml

  backward : softmax_grad

+- op : softplus
+  args : (Tensor x, float beta = 1.0, float threshold = 20.0f)


静态图还是用的 float 类型吗？

这个是保证旧 IR pdmodel 格式模型加载不受影响

参考pr：#72639

zyfncg · 2025-10-13T11:05:06Z

paddle/fluid/pir/serialize_deserialize/0.yaml

+  - op_name : pd_op.softplus
+    actions:
+      - action : modify_attr
+        object : beta
+        type : pir::DoubleAttribute
+        data : 1.0
+      - action : modify_attr
+        object : threshold
+        type : pir::DoubleAttribute
+        data : 20.0


这里和静态图的配置是不是没有对应上？

该修改是为了pir下save/load 版本兼容，具体可见说明：https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/pir/serialize_deserialize/patch/Readme.md

XiaoguangHu01

LGTM

…ional.softplus to double (PaddlePaddle#75426) * fix beta and threshold of Softplus to double * fix test_softplus_activation_fuse_pass v1 * fix test_activation_zero * fix flaot of SoftplusDoubleGradKernel to double * add op_patches for softplus * add yaml for ops/yaml/legacy * fix infershape/operator for FLOAT64 * fix * add SoftPlusOpTranscriber * fix * fix * fix1 * fix2 * fix coverage * fix coverage2

* CallScalarFunction uses the dtype of 'self' as the type of 'other' when opotype is 'div'(#75237) * LinspaceKernel uses the dtype of 'self' as the type of 'step' when tensor is floating (#75238) * align LinspaceKernel * update meta * update gpu kernel * fix LinspaceKernelInner * improve kernel * fix CudaSigmoidGradFunctor and CudaSiluGradFunctor (#75341) * Softplus accuracy and torch alignment 1 (#75363) * [Precision Depth Alignment] paddle.tan reverse calculation: dx = dout *(1 + tan(x)^2) (#75335) * Tan reverse calculation: dx = dout *(1 + tan(x)^2) * [Precision Depth Alignment] Add support for CUDNN to paddle.nn.functional.grid_sample to align with torch accuracy. (#75355) * accuracy_stable_grid_sample * fix * correlation supports big tensor (#75383) * fix * fix test * fix * paddle.tanh Grad and torch alignment (float16) (#75454) * [Precision Depth Alignment] paddle.sin and paddle.cos aligns with torch precision. (#75503) * accuracy_stable_sin * accuracy_stable_cos * [深度对齐]Divide (#75379) * fix * fix * fix * fix * fix * [Precision Depth Alignment] fix precision for float16 of paddle.tan backward (#75525) * fix precision for float16 of paddle.tan backward * fix else branch of CudaTanGradFunctor * [Precision Depth Alignment] fix precision for paddle.expm1 (#75549) * accuracy_stable_expm1 * fix * Bigtensor排查修复[Paddle/paddle/phi/kernels/funcs] (#75523) * fix * fix * [Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double (#75426) * fix beta and threshold of Softplus to double * fix test_softplus_activation_fuse_pass v1 * fix test_activation_zero * fix flaot of SoftplusDoubleGradKernel to double * add op_patches for softplus * add yaml for ops/yaml/legacy * fix infershape/operator for FLOAT64 * fix * add SoftPlusOpTranscriber * fix * fix * fix1 * fix2 * fix coverage * fix coverage2 * fix (#75605) * [深度对齐] dot (#75717) * fix * fix * fix dcu * [Precision Depth Alignment] paddle.log aligns with torch precision (#75799) * accuracy_stable_log * accuracy_stable_log * fix * fix * fix * fix * fix5 * [Precision Depth Alignment] fix eps of paddle.logit from float to double (#75816) * accuracy_stable_logit * add LogitOpTranscriber * fix coverage * fix 0yaml * [Precision Depth Alignment] paddle.log_sigmoid (#75898) * accuracy_stable_log_sigmoid * fix test_activation_stride_op.py * [Precision Depth Alignment] Modify the negative_slope parameter of the paddle.nn.functional.leaky_relu API to double (#75547) * [big tensor] Paddle/paddle/phi/kernels/funcs gpuBigtensor (#75856) * fix funcs * gpu * fix * fix * 修改PADDLE_ENFORCE信息 * fix cpu error * fix dcu * fix dcu * fix * [Fix] log sigmoid complex (#75953) * feature: Add specialized LogSigmoidFunctor and CudaLogSigmoidFunctor for complex numbers This commit introduces specialized implementations of LogSigmoidFunctor and CudaLogSigmoidFunctor to handle complex number inputs. The new implementations utilize direct formulas for improved accuracy and stability in calculations involving complex types. * refactor: Optimize LogSigmoidFunctor and CudaLogSigmoidFunctor for complex types by caching exp(-x) to reduce redundant computations. This change enhances performance while maintaining accuracy in calculations. * refactor: modified the formula in LogSigmoidFunctor to make it numerical stable --------- Co-authored-by: Zhan Rongrui <46243324+zrr1999@users.noreply.github.com> Co-authored-by: 正在学习 <62892980+cszdrg@users.noreply.github.com> Co-authored-by: Bvicii <98971614+scyyh11@users.noreply.github.com>

zhengshengning added 2 commits September 21, 2025 13:11

fix beta and threshold of Softplus to double

9f08428

merge develop

97b7153

zhengshengning requested review from ForFishes, LiYuRio and zhiqiu as code owners September 21, 2025 13:18

zhengshengning mentioned this pull request Sep 21, 2025

[Precision Depth Alignment] paddle.nn.functional.softplus accuracy and torch alignment #75363

Merged

zhengshengning added 3 commits September 23, 2025 10:38

fix test_softplus_activation_fuse_pass v1

f5d6948

fix test_activation_zero

de89ecc

fix flaot of SoftplusDoubleGradKernel to double

b817808

zhengshengning mentioned this pull request Sep 25, 2025

[Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double PaddlePaddle/PaddleCustomDevice#2000

Merged

zhengshengning added 4 commits September 26, 2025 04:56

add op_patches for softplus

17ef802

add yaml for ops/yaml/legacy

2537a9f

fix infershape/operator for FLOAT64

822bbe7

fix

1c0e11c

zhengshengning requested review from DrRyanHuang, JiabinYang, SigureMo, XiaoguangHu01, Xreki, gouzil, phlrain, qili93, sneaxiy, wanghuancoder, xiaoguoguo626807, zhwesky2010 and zrr1999 as code owners October 9, 2025 03:09

zhengshengning force-pushed the accuracy_stable_softplus_v2 branch from d515196 to 1c0e11c Compare October 9, 2025 03:12

zhengshengning removed the request for review from zhiqiu October 9, 2025 03:13

zhengshengning added 5 commits October 9, 2025 08:24

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

7eec903

… accuracy_stable_softplus_v2

fix

f60b944

fix

f36ae90

fix1

7fef327

fix2

1b79191

PaddlePaddle deleted a comment from codecov-commenter Oct 11, 2025

zhengshengning added 2 commits October 11, 2025 11:52

fix coverage

8d6e0a4

fix coverage2

6a3694c

zhengshengning requested review from XiaoguangHu01, luotao1, zhangbo9674 and zyfncg October 13, 2025 02:42

changeyoung98 approved these changes Oct 13, 2025

View reviewed changes

wanghuancoder approved these changes Oct 13, 2025

View reviewed changes

zhangbo9674 approved these changes Oct 13, 2025

View reviewed changes

luotao1 approved these changes Oct 13, 2025

View reviewed changes

Eddie-Wang1120 approved these changes Oct 13, 2025

View reviewed changes

zhengshengning mentioned this pull request Oct 13, 2025

[Precision Depth Alignment] fix eps of paddle.logit from float to double #75816

Merged

zyfncg reviewed Oct 13, 2025

View reviewed changes

zyfncg approved these changes Oct 13, 2025

View reviewed changes

XiaoguangHu01 approved these changes Oct 13, 2025

View reviewed changes

zhengshengning merged commit deed9d3 into PaddlePaddle:develop Oct 13, 2025
91 of 99 checks passed

wanghuancoder mentioned this pull request Oct 24, 2025

[Cherry-pick Fleety_12] Bigtensor and api precision #76023

Closed

zhengshengning mentioned this pull request Oct 24, 2025

[Cherry-pick Fleety_12] Bigtensor and api precision #76028

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double #75426

[Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double #75426

Uh oh!

zhengshengning commented Sep 21, 2025 •

edited

Loading

Uh oh!

paddle-bot bot commented Sep 21, 2025

Uh oh!

codecov-commenter commented Oct 11, 2025 •

edited

Loading

Uh oh!

changeyoung98 left a comment

Uh oh!

wanghuancoder left a comment

Uh oh!

wanghuancoder left a comment

Uh oh!

zyfncg Oct 13, 2025

Uh oh!

zhengshengning Oct 13, 2025

Uh oh!

zhengshengning Oct 13, 2025

Uh oh!

zyfncg Oct 13, 2025

Uh oh!

zhengshengning Oct 13, 2025

Uh oh!

XiaoguangHu01 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

[Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double #75426

[Precision Depth Alignment] fix beta and threshold of paddle.nn.functional.softplus to double #75426

Uh oh!

Conversation

zhengshengning commented Sep 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Category

PR Types

Description

Uh oh!

paddle-bot bot commented Sep 21, 2025

Uh oh!

codecov-commenter commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

changeyoung98 left a comment

Choose a reason for hiding this comment

Uh oh!

wanghuancoder left a comment

Choose a reason for hiding this comment

Uh oh!

wanghuancoder left a comment

Choose a reason for hiding this comment

Uh oh!

zyfncg Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

zhengshengning Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

zhengshengning Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

zyfncg Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

zhengshengning Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

zhengshengning commented Sep 21, 2025 •

edited

Loading

codecov-commenter commented Oct 11, 2025 •

edited

Loading