Description
Anything you want to discuss about vllm.
vLLM heavily depends on PyTorch, and also actively works with PyTorch team to leverage their new features. When a new PyTorch version comes out, vLLM usually upgrades to the latest PyTorch directly.
Meanwhile, vLLM supports diverse hardware backends from different vendors. They often require their own PyTorch versions.
In order to speed up the development of vLLM, hereby we require all vendors to keep up with PyTorch.
Starting from PyTorch 2.5 (Release Day (10/17/24)), vLLM will drop hardware support if it cannot support PyTorch 2.5.
potentially affected vendors and the current PyTorch version they require:
- neuron (2.1.2)
- openvino (2.1.2)
- intel xpu (2.3.1)
Note that latest pytorch support is a necessary condition for vLLM's hardware vendors. It is not sufficient. The vLLM team considers adding new hardware support depending on the community interest, the priority of the main branch, and the bandwidth of the team.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.