加载 qwen微调后的lora 权重 No such device (os error 19) #2021
Description
Reminder
- I have read the README and searched the existing issues.
Reproduction
加载qwen 微调后的lora ，命令：
export USE_MODELSCOPE_HUB=1
CUDA_VISIBLE_DEVICES=0 python src/cli_demo.py
--model_name_or_path qwen/Qwen-7B
--adapter_name_or_path ./output/sft/checkpoint-4000/
--template default
--finetuning_type lora
Expected behavior
12/28/2023 19:02:13 - INFO - llmtuner.model.adapter - Fine-tuning method: LoRA
cuda
./output/sft/checkpoint-4000/adapter_model.safetensors
Traceback (most recent call last):
File "/code/my_chatglm/LLaMA-Factory/src/cli_demo.py", line 47, in
main()
File "/code/my_chatglm/LLaMA-Factory/src/cli_demo.py", line 13, in main
chat_model = ChatModel()
File "/code/my_chatglm/LLaMA-Factory/src/llmtuner/chat/chat_model.py", line 27, in init
self.model, self.tokenizer = load_model_and_tokenizer(
File "/code/my_chatglm/LLaMA-Factory/src/llmtuner/model/loader.py", line 98, in load_model_and_tokenizer
model = init_adapter(model, model_args, finetuning_args, is_trainable)
File "/code/my_chatglm/LLaMA-Factory/src/llmtuner/model/adapter.py", line 81, in init_adapter
model = PeftModel.from_pretrained(model, adapter)
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/peft/peft_model.py", line 352, in from_pretrained
model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs)
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/peft/peft_model.py", line 689, in load_adapter
adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs)
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/peft/utils/save_and_load.py", line 270, in load_peft_weights
adapters_weights = safe_load_file(filename, device=device)
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/safetensors/torch.py", line 312, in load_file
with safe_open(filename, framework="pt", device=device) as f:
OSError: No such device (os error 19)
System Info
System Info
Linux debug-dev7-76fff85c7f-pdxc8 4.18.20-2.el7.wuba.lp.x86_64 huggingface/diffusers#1 SMP Mon Dec 23 17:24:31 CST 2019 x86_64 x86_64 x86_64 GNU/Linux
Package Version
accelerate 0.25.0
addict 2.4.0
aiofiles 23.2.1
aiohttp 3.9.1
aiosignal 1.3.1
aliyun-python-sdk-core 2.14.0
aliyun-python-sdk-kms 2.16.2
altair 5.2.0
annotated-types 0.6.0
anyio 4.2.0
async-timeout 4.0.3
attrs 23.1.0
certifi 2023.11.17
cffi 1.16.0
charset-normalizer 3.3.2
click 8.1.7
contourpy 1.2.0
crcmod 1.7
cryptography 41.0.7
cycler 0.12.1
datasets 2.16.0
dill 0.3.7
docstring-parser 0.15
einops 0.7.0
exceptiongroup 1.2.0
fastapi 0.108.0
ffmpy 0.3.1
filelock 3.13.1
fonttools 4.47.0
frozenlist 1.4.1
fsspec 2023.10.0
gast 0.5.4
gradio 3.50.2
gradio_client 0.6.1
h11 0.14.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.20.1
idna 3.6
importlib-metadata 7.0.1
importlib-resources 6.1.1
jieba 0.42.1
Jinja2 3.1.2
jmespath 0.10.0
joblib 1.3.2
jsonschema 4.20.0
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.8.2
mdurl 0.1.2
modelscope 1.10.0
mpmath 1.3.0
multidict 6.0.4
multiprocess 0.70.15
networkx 3.2.1
nltk 3.8.1
numpy 1.26.2
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
orjson 3.9.10
oss2 2.18.3
packaging 23.2
pandas 2.1.4
peft 0.7.1
Pillow 10.1.0
pip 23.3.1
platformdirs 4.1.0
protobuf 4.25.1
psutil 5.9.7
pyarrow 14.0.2
pyarrow-hotfix 0.6
pycparser 2.21
pycryptodome 3.19.0
pydantic 2.5.3
pydantic_core 2.14.6
pydub 0.25.1
Pygments 2.17.2
pyparsing 3.1.1
python-dateutil 2.8.2
python-multipart 0.0.6
pytz 2023.3.post1
PyYAML 6.0.1
readline 6.2.4.1
referencing 0.32.0
regex 2023.12.25
requests 2.31.0
rich 13.7.0
rouge-chinese 1.0.3
rpds-py 0.15.2
safetensors 0.4.1
scipy 1.11.4
semantic-version 2.10.0
sentencepiece 0.1.99
setuptools 68.2.2
shtab 1.6.5
simplejson 3.19.2
six 1.16.0
sniffio 1.3.0
sortedcontainers 2.4.0
sse-starlette 1.8.2
starlette 0.32.0.post1
sympy 1.12
tiktoken 0.5.2
tokenizers 0.15.0
tomli 2.0.1
toolz 0.12.0
torch 2.1.2
tqdm 4.66.1
transformers 4.36.2
transformers-stream-generator 0.0.4
triton 2.1.0
trl 0.7.7
typing_extensions 4.9.0
tyro 0.6.2
tzdata 2023.3
urllib3 2.1.0
uvicorn 0.25.0
websockets 11.0.3
wheel 0.41.2
xxhash 3.4.1
yapf 0.40.2
yarl 1.9.4
zipp 3.17.0
Others
No response