[Performance]: Query: Memory (VRAM vs. RAM) and Performance Implications of Scaling LoRA Adapters in vLLM

[fighterzzzh]

Proposal to improve performance

I would like to inquire about the resource allocation when deploying multiple LoRA adapters using vLLM. I am using the following command to serve the model:
Generated bash
CUDA_VISIBLE_DEVICES=7 vllm serve /home/gpuserver/Downloads/zzh/Qwen/Qwen2.5-VL-3B-Instruct
--enable-lora
--lora-modules lora1=/path/to/lora/sft lora2=/path/to/lora/sft
Use code with caution.
Bash
My primary question is: as the number of LoRA adapters increases, which memory resource is primarily consumed—GPU memory (VRAM) or system memory (RAM)?
Furthermore, I am curious about the performance. If there is a mechanism that swaps LoRA adapters from system memory to VRAM on demand, can a reasonable level of inference speed still be guaranteed?

Report of performance regression

No response

Misc discussion on performance

No response

Your current environment (if you think it is necessary)

The output of `python collect_env.py`

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[jeejeelee]

Hi, you can refer to: https://github.com/vllm-project/vllm/blob/main/vllm/lora/worker_manager.py

[chenhongyu2048]

I think GPU memory will be consumed first until max num LoRA, and then the excess LoRA will be saved in CPU memory