You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
you should pay attention to the eval_steps,maybe you should set the eval_steps to a large value, like 200000,beacuase in the eval time , swift will return a memory bug so you should set the eval_steps to a very large value.
99
+
2.Lora Tuning:
100
+
101
+
The lora target model are k and v weight in llm you should pay attention to the eval_steps,maybe you should set the eval_steps to a large value, like 200000,beacuase in the eval time , swift will return a memory bug so you should set the eval_steps to a very large value.
94
102
```shell
95
103
# Experimental environment: A100
96
104
# 32GB GPU memory
97
105
CUDA_VISIBLE_DEVICES=0 swift sft \
98
106
--model_type minicpm-v-v2_5-chat \
99
107
--dataset coco-en-2-mini \
100
108
```
101
-
3. all parameters finetune:
102
-
when the argument of lora_target_modules is ALL, the model will finetune all the parameters.
109
+
3. All parameters finetune:
110
+
111
+
When the argument of lora_target_modules is ALL, the model will finetune all the parameters.
103
112
```shell
104
113
CUDA_VISIBLE_DEVICES=0,1 swift sft \
105
114
--model_type minicpm-v-v2_5-chat \
@@ -108,14 +117,17 @@ CUDA_VISIBLE_DEVICES=0,1 swift sft \
108
117
--eval_steps 200000
109
118
```
110
119
111
-
## lora merge and infer
112
-
1. load the lora weight to infer run the follow code:
120
+
## Lora Merge and Infer
121
+
The lora weight can be merge to the base model and then load to infer.
122
+
123
+
1. Load the lora weight to infer run the follow code:
113
124
```shell
114
125
CUDA_VISIBLE_DEVICES=0 swift infer \
115
126
--ckpt_dir /your/lora/save/checkpoint
116
127
```
117
-
2. merge the lora weight to the base model:
118
-
the code will load and merge the lora weight to the base model, save the merge model to the lora save path and load the merge model to infer
128
+
2. Merge the lora weight to the base model:
129
+
130
+
The code will load and merge the lora weight to the base model, save the merge model to the lora save path and load the merge model to infer
0 commit comments