[LoRA] add quick_lora

[JunnYu]

PR types

New features

PR changes

APIs

Description

• 优化lora的前向和反向计算。
• 非动态padding、短文本的情况下，提速约为3%。

已知缺陷：

• lora dropout 必须设置为0

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 29.93197% with 103 lines in your changes are missing coverage. Please review.

Project coverage is 55.41%. Comparing base (f005084) to head (749b419).

Files	Patch %	Lines
paddlenlp/peft/lora/lora_quick_layers.py	28.12%	69 Missing ⚠️
paddlenlp/peft/lora/lora_layers.py	28.88%	32 Missing ⚠️
paddlenlp/peft/lora/lora_config.py	66.66%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8106      +/-   ##
===========================================
- Coverage    55.44%   55.41%   -0.03%     
===========================================
  Files          596      597       +1     
  Lines        91464    91587     +123     
===========================================
+ Hits         50713    50754      +41     
- Misses       40751    40833      +82     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gongel]

冲突了

[lugimzzz]

求input_grad是不是可以考虑使用merged_weight，input_grad= paddle.matmul(grad_output, merged_weight, transpose_y=True)

[JunnYu]

merged_weight这个东西没法从前向复用，复用会占用很大的显存。
然后如果合并计算的话，就无法复用 lora_B_input_grad = paddle.matmul(grad_output, lora_B, transpose_y=True)。需要重新计算一次

[lugimzzz]

lgtm