Skip to content

Commit 96bd6b9

Browse files
按照要求修改了标题,术语,章节描述
1 parent 432cca9 commit 96bd6b9

File tree

1 file changed

+29
-17
lines changed

1 file changed

+29
-17
lines changed

docs/swift_train_and_infer.md

+29-17
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,25 @@
1-
## swift install
1+
## Swift install
2+
You can quickly install Swift using bash commands.
3+
24
``` bash
35
git clone https://github.com/modelscope/swift.git
46
cd swift
57
pip install -r requirements.txt
68
pip install -e '.[llm]'
79
```
810

9-
## Swift infer
10-
### quick start
11-
1. run the bash code will download the model of MiniCPM-Llama3-V-2_5 and run the inference
11+
## Swift Infer
12+
Inference using Swift can be carried out in two ways: through a command line interface and via Python code.
13+
14+
### Quick start
15+
Here are the steps to launch Swift from the Bash command line:
16+
17+
1. Run the bash code will download the model of MiniCPM-Llama3-V-2_5 and run the inference
1218
``` shell
1319
CUDA_VISIBLE_DEVICES=0 swift infer --model_type minicpm-v-v2_5-chat
1420
```
1521

16-
2. you can also run the code with more arguments below to run the inference:
22+
2. You can also run the code with more arguments below to run the inference:
1723
```
1824
model_id_or_path # 可以写huggingface的模型id或者本地模型地址
1925
infer_backend ['AUTO', 'vllm', 'pt'] # 后段推理,默认auto
@@ -30,14 +36,15 @@ CUDA_VISIBLE_DEVICES=0 swift infer --model_type minicpm-v-v2_5-chat
3036
quant_method ['bnb', 'hqq', 'eetq', 'awq', 'gptq', 'aqlm'] # 模型的量化方式
3137
quantization_bit [0, 1, 2, 3, 4, 8] 默认是0,代表不使用量化
3238
```
33-
3. example:
39+
3. Example:
3440
``` shell
3541
CUDA_VISIBLE_DEVICES=0,1 swift infer \
3642
--model_type minicpm-v-v2_5-chat \
3743
--model_id_or_path /root/ld/ld_model_pretrain/MiniCPM-Llama3-V-2_5 \
3844
--dtype bf16
3945
```
40-
### python code with swift infer
46+
### Python code with swift infer
47+
The following demonstrates using Python code to initiate inference with the MiniCPM-Llama3-V-2_5 model through Swift.
4148

4249
```python
4350
import os
@@ -82,24 +89,26 @@ CUDA_VISIBLE_DEVICES=0 swift infer --model_type minicpm-v-v2_5-chat
8289
```
8390

8491
## Swift train
85-
1. make the train data like this:
92+
Swift supports training on the local dataset,the training steps are as follows:
93+
1. Make the train data like this:
8694
```jsonl
8795
{"query": "这张图片描述了什么", "response": "这张图片有一个大熊猫", "images": ["local_image_path"]}
8896
{"query": "这张图片描述了什么", "response": "这张图片有一个大熊猫", "history": [], "images": ["image_path"]}
8997
{"query": "竹子好吃么", "response": "看大熊猫的样子挺好吃呢", "history": [["这张图有什么", "这张图片有大熊猫"], ["大熊猫在干嘛", "吃竹子"]], "images": ["image_url"]}
9098
```
91-
2. lora turning:
92-
the lora target model are k and v weight in llm
93-
you should pay attention to the eval_steps, maybe you should set the eval_steps to a large value, like 200000,beacuase in the eval time , swift will return a memory bug so you should set the eval_steps to a very large value.
99+
2. Lora Tuning:
100+
101+
The lora target model are k and v weight in llm you should pay attention to the eval_steps,maybe you should set the eval_steps to a large value, like 200000,beacuase in the eval time , swift will return a memory bug so you should set the eval_steps to a very large value.
94102
```shell
95103
# Experimental environment: A100
96104
# 32GB GPU memory
97105
CUDA_VISIBLE_DEVICES=0 swift sft \
98106
--model_type minicpm-v-v2_5-chat \
99107
--dataset coco-en-2-mini \
100108
```
101-
3. all parameters finetune:
102-
when the argument of lora_target_modules is ALL, the model will finetune all the parameters.
109+
3. All parameters finetune:
110+
111+
When the argument of lora_target_modules is ALL, the model will finetune all the parameters.
103112
```shell
104113
CUDA_VISIBLE_DEVICES=0,1 swift sft \
105114
--model_type minicpm-v-v2_5-chat \
@@ -108,14 +117,17 @@ CUDA_VISIBLE_DEVICES=0,1 swift sft \
108117
--eval_steps 200000
109118
```
110119

111-
## lora merge and infer
112-
1. load the lora weight to infer run the follow code:
120+
## Lora Merge and Infer
121+
The lora weight can be merge to the base model and then load to infer.
122+
123+
1. Load the lora weight to infer run the follow code:
113124
```shell
114125
CUDA_VISIBLE_DEVICES=0 swift infer \
115126
--ckpt_dir /your/lora/save/checkpoint
116127
```
117-
2. merge the lora weight to the base model:
118-
the code will load and merge the lora weight to the base model, save the merge model to the lora save path and load the merge model to infer
128+
2. Merge the lora weight to the base model:
129+
130+
The code will load and merge the lora weight to the base model, save the merge model to the lora save path and load the merge model to infer
119131
```shell
120132
CUDA_VISIBLE_DEVICES=0 swift infer \
121133
--ckpt_dir your/lora/save/checkpoint \

0 commit comments

Comments
 (0)