Skip to content

【开源实习】blenderbot_small模型微调 #1980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

outbreak-sen
Copy link
Contributor

分别实现了blenderbot_small模型在"stanfordnlp/coqa"和"google/Synthetic-Persona-Chat"数据集上的微调实验。
因为在coqa数据集上的效果并不好,所以又做了Persona的微调,Persona的微调还是不错的,这里都上传了供参考
任务链接在https://gitee.com/mindspore/community/issues/IAUPE8
transformers+pytorch+3090的benchmark是自己编写的,仓库位于https://github.com/outbreak-sen/blenderbot_small_finetuned
更改代码位于llm/finetune/blenderbot_small,只包含mindnlp+mindspore的
实验结果如下

Blenderbot_Small的coqa微调

硬件

资源规格:NPU: 1*Ascend-D910B(显存: 64GB), CPU: 24, 内存: 192GB

智算中心:武汉智算中心

镜像:mindspore_2_5_py311_cann8

torch训练硬件资源规格:Nvidia 3090

模型与数据集

模型:"facebook/blenderbot_small-90M"

数据集:"stanfordnlp/coqa"

训练损失

trainloss mindspore+mindnlp Pytorch+transformers
1 0.0117 0.3391
2 0.0065 0.0069
3 0.0041 0.0035
4 0.0027
5 0.0017
6 0.0012
7 0.0007
8 0.0005
9 0.0003
10 0.0002

评估损失

eval loss mindspore+mindnlp Pytorch+transformers
1 0.010459424927830696 0.010080045089125633
2 0.010958473198115826 0.008667134679853916
3 0.011061458848416805 0.00842051301151514
4 0.011254088021814823 0.00842051301151514
5 0.011891312897205353
6 0.012321822345256805
7 0.012598296627402306
8 0.01246054656803608
9 0.0124361552298069
10 0.01264810748398304

Blenderbot_Small的Synthetic-Persona-Chat微调

硬件

资源规格:NPU: 1*Ascend-D910B(显存: 64GB), CPU: 24, 内存: 192GB

智算中心:武汉智算中心

镜像:mindspore_2_5_py311_cann8

torch训练硬件资源规格:Nvidia 3090

模型与数据集

模型:"facebook/blenderbot_small-90M"

数据集:"google/Synthetic-Persona-Chat"

训练损失

trainloss mindspore+mindnlp Pytorch+transformers
1 0.1737 0.2615
2 0.1336 0.1269
3 0.1099 0.0987

评估损失

eval loss mindspore+mindnlp Pytorch+transformers
1 0.16312436759471893 0.160710409283638
2 0.15773458778858185 0.15692724287509918
3 0.15398454666137695 0.1593361645936966
4 0.15398454666137695 0.1593361645936966

@lvyufeng lvyufeng merged commit 2a09101 into mindspore-lab:master Mar 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants