add fp8linear

[wanderHZ]

PR types

New features

PR changes

APIs

Description

qat-fp8linear

[paddle-bot]

Thanks for your contribution!

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

liukebin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[lugimzzz]

if act_scale is not None:
        if training:
            scale = paddle.max(paddle.abs(target_x)) / qmax + quantization_config.epsilon
            if state < quantization_config.skip_first_act_scale_step:
                act_scale.set_value((state * act_scale + scale) / (state + 1))
            else:
                act_scale.set_value((1-quantization_config.moving_rate)*act_scale + quantization_config.moving_rate*scale)
                scale = act_scale

        else:
            scale = act_scale
    else:
        scale = paddle.max(paddle.abs(target_x)) / qmax + quantization_config.epsilon
``` 量化scale统计建议改成这种方式，之前的方式实验的时候发现会训练一段时间后突刺或者loss就不收敛了