[Feature] Enable vLLM V1 feature and Tensor/Pipeline Parallel to improve Performance

[louie-tsai]

Priority

P3-Medium

OS type

Ubuntu

Hardware type

Xeon-EMR

Running nodes

Single Node

Description

To improve vLLM performance on Xeon, we need to use Tensor Parallel/Pipeline Parallel and V1 feature to improve performance on Xeon. potentially, we might have 2-3X speedup

[louie-tsai]

This feature depends on two below PRs:
vllm-project/vllm#17930
vllm-project/vllm#19241

[yinghu5]

also link to #2061

[louie-tsai]

Due to no vllm-project/vllm#19241 in the latest 0.9.1 release, we need to wait for 0.9.2 release for the feature.