Skip to content

Upstream merge 2025 04 25 #524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 82 commits into from
Apr 25, 2025
Merged

Upstream merge 2025 04 25 #524

merged 82 commits into from
Apr 25, 2025

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Apr 25, 2025

Including V1 performance and Deepseek fix

njhill and others added 30 commits April 22, 2025 23:27
@njhill
[BugFix] Remove default multiproc executor collective_rpc timeout (v…
5175b88 
…llm-project#17000)

Signed-off-by: Nick Hill <nhill@redhat.com>
@Chenyaaang
[Core][V1][TPU] Enable structured decoding on TPU V1 (vllm-project#16499
83d9337 
)

Signed-off-by: Chenyaaang <chenyangli@google.com>
@gcalmettes
[Bugfix] validate urls object for multimodal content parts (vllm-proj…
36fe787 
…ect#16990)

Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
@yangw-dev
add Dockerfile build vllm against torch nightly (vllm-project#16936)
f67e9e9 
Signed-off-by: Yang Wang <elainewy@meta.com>
@maleksan85 @SageMoore @qli88
[Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (vllm-pro…
bc7c4d2 
…ject#13305)

Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Signed-off-by: maleksan85 <maleksan@amd.com>
Signed-off-by: <>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
@njhill
[V1][DP] More robust DP/EP dummy request coordination (vllm-project#1…
1e013fa 
…6277)

Signed-off-by: Nick Hill <nhill@redhat.com>
@vllmellm
[BugFix] Revert ROCm Custom Paged Attention Env Flag Check (vllm-proj…
7e081ba 
…ect#17022)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
@chaunceyjiang
Revert "[Misc] Add S3 environment variables for better support of Min…
6bc1e30 
…IO." (vllm-project#17021)
@youkaichao
[misc] tune some env vars for GB200 (vllm-project#16992)
e1cf90e 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@xuechendi @madamczyk-intel
[INTEL-HPU][v0] Port delayed sampling to upstream (vllm-project#16949)
56a7352 
Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
@reidliu41
[doc] add download path tips (vllm-project#17013)
eb8ef42 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@vllmellm
[Bugfix] Triton FA function takes no keyword arguments (vllm-project#…
047797e 
…16902)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
@njhill
[V1] Avoid socket errors during shutdown when requests are in in-flig…
b2f195c 
…ht (vllm-project#16807)

Signed-off-by: Nick Hill <nhill@redhat.com>
@LucasWilkinson
[BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have …
d0da99f 
…shape (metadata_size) (vllm-project#16998)

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
@gitover22
[Misc] Improve readability of get_open_port function. (vllm-project#1…
ec69124 
…7024)

Signed-off-by: gitover22 <qidizou88@gmail.com>
@chaunceyjiang
[Bugfix] Fix AssertionError: skip_special_tokens=False is not support…
8c87a9a 
…ed for Mistral tokenizers (vllm-project#16964)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@russellb
[CI] Run v1/test_serial_utils.py in CI (vllm-project#16996)
ce17db8 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@mgoin
Mistral-format support for compressed-tensors (vllm-project#16803)
aa72d9a 
Signed-off-by: mgoin <mgoin64@gmail.com>
@mgoin
Categorize tests/kernels/ based on kernel type (vllm-project#16799)
6317a51 
Signed-off-by: mgoin <mgoin64@gmail.com>
@windsonsea
[Doc] Add top anchor and a note to quantization/bitblas.md (vllm-proj…
f7912cb 
…ect#17042)

Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
@hmellor
Ensure that pid passed to kill_process_tree is int for mypy (v…
53c0fa1 
…llm-project#17051)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@russellb
[CI] Update structured-output label automation (vllm-project#17055)
af869f6 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@hmellor
Improve Transformers backend model loading QoL (vllm-project#17039)
8e630d6 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor
CacheConfig.block_size should always be int when used (vllm-proje…
f3a21e9 
…ct#17052)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor
Use @property and private field for data_parallel_rank_local (vll…
bdb3660 
…m-project#17053)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@tjohnson31415
[Frontend] Support guidance:no-additional-properties for compatibilit…
3cde34a 
…y with xgrammar (vllm-project#15949)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
@sarckk
[BugFix][V1] Fix int32 token index overflow when preparing input ids (v…
32d4b66 
…llm-project#16806)
@WoosukKwon
[V1][Spec Decode] Always use argmax for sampling draft tokens (vllm-p…
41fb013 
…roject#16899)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@csy1204 @mgoin
[CI/Build] workaround for CI build failure (vllm-project#17070)
b07d741 
Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@CXIAAAAA
[Quantization]add prefix for commandA quantized model (vllm-project#1…
6b2427f 
…7017)
mgoin and others added 15 commits April 25, 2025 04:20
@mgoin
[Docs] Fix True->true in supported_models.md (vllm-project#17141)
6498189
@hmellor
Move missed SchedulerConfig args into scheduler config group in `En…
6ca0234 
…gineArgs` (vllm-project#17131)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@lifuhuang
[Misc] Clean up redundant code in uniproc_executor.py (vllm-project#1…
5aa6efb 
…6762)

Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
@MengqingCao
[Bugfix][Misc] Use TritonPlaceholderModule to defensively import trit…
2f54045 
…on (vllm-project#15099)

Signed-off-by: Mengqing Cao <cmq0113@163.com>
@LucasWilkinson
[Misc] Benchmark Serving Script Support Appending Results (vllm-proje…
881f735 
…ct#17028)

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
@cynthieye @MagnetoWang
[Perf]Optimize rotary_emb implementation to use Triton operator for i…
b22980a 
…mproved inference performance (vllm-project#16457)

Signed-off-by: cynthieye <yexin93@qq.com>
Co-authored-by: MagnetoWang <magnetowang@outlook.com>
@csy1204
[Bugfix] remove fallback in guided_json (int range, patterns) (vllm-p…
6aae216 
…roject#16725)

Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>
@rasmith @ProExpertProg
[Quantization][FP8] Add support for FP8 models with input_scale for o…
a41351f 
…utput projection and QK quantization (vllm-project#15734)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
@windsonsea
[Doc] Add headings to improve gptqmodel.md (vllm-project#17164)
ef19e67 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
@houseroad
Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (vl…
fc966e9 
…lm-project#17158)
@windsonsea
[Doc] Add two links to disagg_prefill.md (vllm-project#17168)
f851b84 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
@alex-jw-brooks
[Doc] Move todo out of beam search docstring (vllm-project#17183)
7feae92 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@DarkLight1337
[Bugfix] Fix mistral model tests (vllm-project#17181)
19dcc02 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@JasmondL @DarkLight1337
[Bugfix] Fix Mistral ChatCompletionRequest Body Exception (vllm-proje…
d5615af 
…ct#16769)

Signed-off-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@gshtras
Merge remote-tracking branch 'upstream/main'
8c211e5
shajrawi
shajrawi previously approved these changes Apr 25, 2025
View reviewed changes
@gshtras gshtras requested a review from shajrawi April 25, 2025 18:57
@gshtras gshtras merged commit 28007b0 into main Apr 25, 2025
2 of 4 checks passed
@gshtras gshtras deleted the upstream_merge_2025_04_25 branch April 25, 2025 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

@Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD Awaiting requested review from Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD is a code owner

@maleksan85 maleksan85 Awaiting requested review from maleksan85 maleksan85 is a code owner

@sunway513 sunway513 Awaiting requested review from sunway513 sunway513 is a code owner

@hongxiayang hongxiayang Awaiting requested review from hongxiayang hongxiayang is a code owner

@divakar-amd divakar-amd Awaiting requested review from divakar-amd divakar-amd is a code owner

@charlifu charlifu Awaiting requested review from charlifu charlifu is a code owner

@mawong-amd mawong-amd Awaiting requested review from mawong-amd mawong-amd is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.