finetune.md

Fine-tune the pretrained LLMs

QLoRA Fine-tune InternLM

• InternLM-7B, oasst1

  xtuner train internlm_7b_qlora_oasst1_e3

• InternLM-7B, Arxiv Gentitle

  xtuner train internlm_7b_qlora_arxiv_gentitle_e3

• InternLM-7B, Colorist

  xtuner train internlm_7b_qlora_colorist_e5

• InternLM-7B, Coder

  xtuner train internlm_7b_qlora_code_alpaca_e3

• InternLM-7B, SQL

  xtuner train internlm_7b_qlora_sql_e3

• InternLM-7B, Lawyer

  xtuner train internlm_7b_qlora_lawyer_e3

• InternLM-7B, Open-Platypus

  xtuner train internlm_7b_qlora_open_platypus_e3

• InternLM-7B, Alpaca-enzh

  xtuner train internlm_7b_qlora_alpaca_enzh_e3

QLoRA Fine-tune Llama2

Don't forget to use huggingface-cli login and input your access token first to access Llama2! See here to learn how to obtain your access token.

• Llama2-7B, MOSS-003-SFT (plugins!)

  NPROC_PER_NODE=8 xtuner train llama2_7b_qlora_moss_sft_all_e2_gpu8  # Recommended!
  xtuner train llama2_7b_qlora_moss_sft_all_e1

• Llama2-7B, Arxiv Gentitle

  xtuner train llama2_7b_qlora_arxiv_gentitle_e3

• Llama2-7B, Colorist

  xtuner train llama2_7b_qlora_colorist_e5

QLoRA Fine-tune Qwen

• Qwen-7B, MOSS-003-SFT (plugins!)

  NPROC_PER_NODE=8 xtuner train qwen_7b_qlora_moss_sft_all_e2_gpu8  # Recommended!
  xtuner train qwen_7b_qlora_moss_sft_all_e1

• Qwen-7B, oasst1

  xtuner train qwen_7b_qlora_oasst1_e3

• Qwen-7B, Arxiv Gentitle

  xtuner train qwen_7b_qlora_arxiv_gentitle_e3

• Qwen-7B, Alpaca-enzh

  xtuner train qwen_7b_qlora_alpaca_enzh_e3

QLoRA Fine-tune Baichuan

• Baichuan-7B, oasst1

  xtuner train baichuan_7b_qlora_oasst1_e3

• Baichuan-7B, Arxiv Gentitle

  xtuner train baichuan_7b_qlora_arxiv_gentitle_e3

• Baichuan-7B, Alpaca-enzh

  xtuner train baichuan_7b_qlora_alpaca_enzh_e3