Dropout+layerwisedecay

[sljlp]

PR types

PR changes

Describe

1. 参考add layerwise learning rate for adamw #35569
   paddle.optimizer.Adamw 里有个参数lr_ration, 相当于layerwise_lr_scale
2. 参考[hybrid enhance] add flag to control the avg position for grad merge under pipeline mode #36384
   在num_pp>2时，设置DistributedStrategy.gradient_scale_configs["scale_gradient"] = True, 可以与单卡对的更齐，用法如下：

strategy = paddle.distributed.fleet.DistributedStrategy()
strategy.gradient_scale_configs = {
    'scale_strategy': 'avg',
    'scale_gradient': True
}

[paddle-bot-old]

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.