Skip to content

Issues: vllm-project/vllm

[Roadmap] vLLM Roadmap Q4 2024
#9006 opened Oct 1, 2024 by simon-mo
Open 4
vLLM's V2 Engine Architecture
#8779 opened Sep 24, 2024 by simon-mo
Open 7
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1,715 Open 3,103 Closed
1,715 Open 3,103 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug]: vllm crashes when preemption of priority scheduling is triggered on vllm-0.6.3.dev173+g36ea7907.d20241011 bug Something isn't working
#9342 opened Oct 14, 2024 by tonyaw
1 task done
[Bug]: LLAMA 3.2 11B Vision Instruct Model not Running in VLLM 0.6.2 bug Something isn't working
#9341 opened Oct 14, 2024 by saikatscalers
1 task done
[Installation]: Adding opentelemetry packages in container image installation Installation problems
#9340 opened Oct 14, 2024 by sanketsudake
1 task done
[Usage]: --cpu-offload-gb no use usage How to use vllm
#9339 opened Oct 14, 2024 by Rane2021
1 task done
1
[Feature]: Allow max_tokens = 0 feature request
#9336 opened Oct 14, 2024 by fgebhart
1 task done
[Bug]: missing 'Finished request xxxx' log bug Something isn't working
#9335 opened Oct 14, 2024 by jinzhen-lin
[Bug]: TPU single-host v5e-8 HBM OOM with Llama 3.1 70B and tpu_int8 quantization bug Something isn't working
#9331 opened Oct 14, 2024 by samos123
1 task done
QWEN2-VL Model Inference bug Something isn't working
#9330 opened Oct 14, 2024 by will-wiki
1 task done
1
[Bug]: Exception in worker VllmWorkerProcess while processing method init_device: NCCL error: unhandled cuda error bug Something isn't working
#9329 opened Oct 14, 2024 by wangyao123456a
1 task done
1
[Bug]: Gemma 27B Produces no Outputs (2B and 9B work fine) bug Something isn't working
#9326 opened Oct 13, 2024 by RonanKMcGovern
1 task done
[Feature]: Quantization support for LLaVA OneVision feature request
#9324 opened Oct 13, 2024 by salvaba94
1 task done
1
[Feature]: Support for rhymes-ai/Aria feature request
#9323 opened Oct 13, 2024 by engchina
1 task done
1
[Misc]: remove dropout related stuff from triton flash attention kernel misc
#9322 opened Oct 13, 2024 by HaiShaw
1 task done
[Bug]: vLLM was installed and used without issues, but recently, during more frequent usage, it suddenly throws an error on a particular request and stops working entirely. Even nvidia-smi cannot return any output. The log is as follows: bug Something isn't working
#9321 opened Oct 13, 2024 by alexchenyu
1 task done
[Bug]: 当vLLM 部署实现 OpenAI API，并且生成模型使用llama 3 8b instruct做RAG任务时，模型生成不停 bug Something isn't working
#9320 opened Oct 13, 2024 by asilverlight
1 task done
2
[Bug]: Installed vllm successfully for AMD MI60 but inference is failing bug Something isn't working
#9319 opened Oct 13, 2024 by Said-Akbar
1 task done
[Usage]: [rank0]: AttributeError: 'LLMEngine' object has no attribute 'driver_worker' usage How to use vllm
#9318 opened Oct 13, 2024 by xuyuemei
1 task done
1
[Bug]: KeyError during loading of Mixtral 8x22B in FP8 bug Something isn't working
#9316 opened Oct 12, 2024 by IowaSovereign
1 task done
[help wanted]: write tests for python-only development misc
#9315 opened Oct 12, 2024 by youkaichao
1 task done
1
[RFC]: Let every model be a reward model/embedding model for PRMs RFC
#9314 opened Oct 12, 2024 by zhuzilin
1 task done
2
[Bug]: different generation result when changing parameters using copy_ and = method bug Something isn't working
#9313 opened Oct 12, 2024 by hxdtest
1 task done
[Bug]: api_server.py: error: argument --tool-call-parser: invalid choice: 'llama3_json' (choose from 'mistral', 'hermes') bug Something isn't working
#9312 opened Oct 12, 2024 by joestein-ssc
1 task done
2
[Bug]: Process group watchdog thread terminated with exception: CUDA error: an illegal memory access was encountered bug Something isn't working
#9308 opened Oct 12, 2024 by eyuansu62
1 task done
[Bug]: latest docker build (0.6.2) got error due to VLLM_MAX_SIZE_MB bug Something isn't working
#9307 opened Oct 12, 2024 by ZJLi2013
1 task done
1
[Bug]: Failed to pickle inputs of failed execution: CUDA error: an illegal memory access was encountered bug Something isn't working
#9306 opened Oct 12, 2024 by Clint-chan
1 task done
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.