Fix interleaved_sliding_window config #8

amogkam · 2025-06-16T20:01:18Z

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Signed-off-by: raushan <raushan@huggingface.co>

…ject#19086) Signed-off-by: Lu Fang <lufang@fb.com>

Signed-off-by: nicklucche <nlucches@redhat.com>

Signed-off-by: googs1025 <googs1025@gmail.com>

Signed-off-by: simon-mo <simon.mo@hey.com>

Co-authored-by: Soren Dreano <soren@numind.ai>

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

Signed-off-by: Varun <vsundarr@redhat.com> Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>

…ect#18844) Signed-off-by: mgoin <mgoin64@gmail.com>

…m-project#19093) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Yikun Jiang <yikun@apache.org>

…8654) Signed-off-by: Chen Zhang <zhangch99@outlook.com>

…th prompt fix (vllm-project#18971)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

…hen Thinking is enabled (vllm-project#19075) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: nicklucche <nlucches@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

…for models with sliding window layers (vllm-project#19029) Signed-off-by: Chen Zhang <zhangch99@outlook.com>

…llm-project#18919)

Signed-off-by: Russell Bryant <rbryant@redhat.com> Co-authored-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

…roject#18437)

…t#19059) Signed-off-by: calvin chen <120380290@qq.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>

Signed-off-by: Siyuan Liu <lsiyuan@google.com>

…m-project#19113) Signed-off-by: Seiji Eicher <seiji@anyscale.com>

…9089) Signed-off-by: Isotr0py <2037008807@qq.com>

… divisibility check (vllm-project#19339) Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>

) Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

…lm-project#19649) Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Lu Fang <lufang@fb.com>

…d ComputedBlocksTracker (vllm-project#18957) Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn> Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>

Signed-off-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai> Co-authored-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai> Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>

…ength from tokenizer config (vllm-project#19660) Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>

Signed-off-by: Chengji Yao <chengjiyao@google.com> Co-authored-by: mgoin <mgoin64@gmail.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

Signed-off-by: Russell Bryant <rbryant@redhat.com>

) Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

…llm-project#19626) Signed-off-by: Nick Hill <nhill@redhat.com>

…9557)

…rts. (vllm-project#19652) Signed-off-by: Shawn Tan <shawntan@ibm.com>

) Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

…oject#19662) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: Bill Nell <bnell@redhat.com>

…h MiniMaxText01ForCausalLM) (vllm-project#19677) Signed-off-by: QscQ <qscqesze@gmail.com>

…19446) Signed-off-by: Russell Bryant <rbryant@redhat.com>

Signed-off-by: Amog Kamsetty <amogkamsetty@gmail.com>

github-actions · 2025-06-16T20:01:29Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

…g-attn Signed-off-by: Amog Kamsetty <amogkamsetty@gmail.com>

Signed-off-by: Amog Kamsetty <amogkamsetty@gmail.com>

zucchini-nlp and others added 30 commits June 3, 2025 15:06

[Bugfix] disable processor cache (vllm-project#19068)

d81edde

Signed-off-by: raushan <raushan@huggingface.co>

[Doc] Improve the Pull Request template with key components (vllm-pro…

d00dd65

…ject#19086) Signed-off-by: Lu Fang <lufang@fb.com>

[Misc] Add missing _Backend enums (vllm-project#19081)

4b7817c

Signed-off-by: nicklucche <nlucches@redhat.com>

[Misc] fix: add miss best_of param validation (vllm-project#18555)

d054da1

Signed-off-by: googs1025 <googs1025@gmail.com>

[Misc] Add SPDX-FileCopyrightText (vllm-project#19100)

02f0c7b

Signed-off-by: simon-mo <simon.mo@hey.com>

[Doc] Readme standardization (vllm-project#18695)

19bdaf3

Co-authored-by: Soren Dreano <soren@numind.ai>

[doc] update docker version (vllm-project#19074)

01eee40

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Kernel] DeepEP dispatch-combine kernel integration (vllm-project#18434)

fa98d77

Signed-off-by: Varun <vsundarr@redhat.com> Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

[V1] Support cross-layer KV sharing (vllm-project#18212)

bdf1396

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>

[Perf] Tune scaled_fp8_quant by increasing vectorization (vllm-proj…

e31446b

…ect#18844) Signed-off-by: mgoin <mgoin64@gmail.com>

Fix interaction between Optional and Annotated in CLI typing (vll…

6865fe0

…m-project#19093) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Yikun Jiang <yikun@apache.org>

[v1] Re-init input batch for multiple kv cache groups (vllm-project#1…

6cac54f

…8654) Signed-off-by: Chen Zhang <zhangch99@outlook.com>

[V1][Spec Decode][Ngram] 1.35x gain -> 1.95x gain on InstructCoder wi…

135cf55

…th prompt fix (vllm-project#18971)

[Bugfix] get_num_blocks_to_allocate with null_block (vllm-project#19031)

b5fd950

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

[Bugfix]: Fix the incompatibility issue with tool_choice 'required' w…

4de790f

…hen Thinking is enabled (vllm-project#19075) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

[Bugfix][P/D] Fix Prefix Cache Bug (vllm-project#18411)

5d96533

Signed-off-by: nicklucche <nlucches@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory …

a8da78e

…for models with sliding window layers (vllm-project#19029) Signed-off-by: Chen Zhang <zhangch99@outlook.com>

feat: add data parallel rank to KVEventBatch (vllm-project#18925)

b712be9

[Misc] Fix path and python alias errors in disagg_prefill exmaples (v…

abd7df2

…llm-project#18919)

[Docs] Add developer doc about CI failures (vllm-project#18782)

52dceb1

Signed-off-by: Russell Bryant <rbryant@redhat.com> Co-authored-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

[CPU] V1 support for the CPU backend (vllm-project#16441)

4555143

[Core] Cast multimodal input in hf processor (vllm-project#18862)

1409ef9

Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

[KERNEL] Sampler. CUDA kernel for applying repetition penalty (vllm-p…

5d6d1ad

…roject#18437)

[Cleanup][v1]:remote guided-decoding-backend for example (vllm-projec…

8d646c2

…t#19059) Signed-off-by: calvin chen <120380290@qq.com>

[NVIDIA] Add Cutlass MLA backend (vllm-project#17625)

41aa578

[Bugfix] Fix FA3 full cuda graph correctness (vllm-project#19106)

b124e10

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Fix vllm-project#19130 (vllm-project#19132)

3336c8c

Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>

[TPU] Skip hanging tests (vllm-project#19115)

8e972d9

Signed-off-by: Siyuan Liu <lsiyuan@google.com>

Fix ValueError: Missing value for tag key(s): model_name,engine. (vll…

2669a0d

…m-project#19113) Signed-off-by: Seiji Eicher <seiji@anyscale.com>

[Misc] Add packages for benchmark as extra dependency (vllm-project#1…

8711bc5

…9089) Signed-off-by: Isotr0py <2037008807@qq.com>

22quinn and others added 25 commits June 15, 2025 13:43

[Kernel] Raise verbose error and consolidate num_heads/num_kv_heads…

0b73736

… divisibility check (vllm-project#19339) Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>

[Benchmark] Refactor benchmark script for fp8 & int8 (vllm-project#19627

3d330c4

) Signed-off-by: yewentao256 <zhyanwentao@126.com>

Enable prefix caching with full cuda graphs (vllm-project#19617)

055915e

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

[CI/Build] Fix torch nightly CI dependencies part 2 (vllm-project#19589)

91b2c17

[Misc] Remove duplicate multiproc method setting for CPU platform (vl…

a5e7242

…lm-project#19649) Signed-off-by: Isotr0py <2037008807@qq.com>

[MISC] Remove unused variableds in C++ (vllm-project#19609)

c6703d1

Signed-off-by: Lu Fang <lufang@fb.com>

[Bugfix][Core] Prefix caching causes incorrect outputs due to outdate…

92183b4

…d ComputedBlocksTracker (vllm-project#18957) Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn> Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>

[Misc][Frontend] passthrough bad_words (vllm-project#19564)

367871a

Signed-off-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai> Co-authored-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai> Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>

[Misc] Fix skipped max-model-len validation when deriving max model l…

b692e9c

…ength from tokenizer config (vllm-project#19660) Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>

[TPU] support attention head dim smaller than 128 (vllm-project#19620)

a77aea5

Signed-off-by: Chengji Yao <chengjiyao@google.com> Co-authored-by: mgoin <mgoin64@gmail.com>

[MISC] typo fix (vllm-project#19672)

26bc46e

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[CI] Add mteb testing for rerank models (vllm-project#19344)

f40f763

[Docs] Move multiproc doc to v1 dir (vllm-project#19651)

8d12070

Signed-off-by: Russell Bryant <rbryant@redhat.com>

[Kernel] GGUF MMVQ kernel for multiple input vectors (vllm-project#18754

dec66d2

) Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

[BugFix] Don't catch BaseException when dumping execute_model errors (v…

ee35e96

…llm-project#19626) Signed-off-by: Nick Hill <nhill@redhat.com>

[DOC] Add reasoning capability to vLLM streamlit code (vllm-project#1…

3e75069

…9557)

[Feature]:Allow for Granite MoE Hybrid models with _only_ shared expe…

4d54240

…rts. (vllm-project#19652) Signed-off-by: Shawn Tan <shawntan@ibm.com>

[Bugfix] Fix TP inference for Flex attention backend (vllm-project#19657

1173804

) Signed-off-by: Isotr0py <2037008807@qq.com>

[MISC] bump huggingface_hub pkg to 0.33.0 (vllm-project#19547)

c3fec47

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[Bugfix] fix missing 'finish_reason': null in streaming chat (vllm-pr…

836d4ce

…oject#19662) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

[Kernels] Use empty for modular MoE workspaces (vllm-project#19667)

5e5baa9

Signed-off-by: Bill Nell <bnell@redhat.com>

[Model] Add support for MiniMaxM1ForCausalLM (shares architecture wit…

387bdf0

…h MiniMaxText01ForCausalLM) (vllm-project#19677) Signed-off-by: QscQ <qscqesze@gmail.com>

[V1] Change return type on get_multimodal_embeddings() (vllm-project#…

90f9c2e

…19446) Signed-off-by: Russell Bryant <rbryant@redhat.com>

Merge branch 'main' of github.com:vllm-project/vllm

eb059d5

fix

71e6fb1

Signed-off-by: Amog Kamsetty <amogkamsetty@gmail.com>

amogkam requested a review from simon-mo as a code owner June 16, 2025 20:01

amogkam added 2 commits June 16, 2025 20:11

Merge branch 'main' of github.com:character-tech/vllm into fix-slidin…

274c7d7

…g-attn Signed-off-by: Amog Kamsetty <amogkamsetty@gmail.com>

remove logging

882e4ea

Signed-off-by: Amog Kamsetty <amogkamsetty@gmail.com>

amogkam merged commit 3def9c4 into main Jun 16, 2025
0 of 2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix interleaved_sliding_window config #8

Fix interleaved_sliding_window config #8

Uh oh!

amogkam commented Jun 16, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Jun 16, 2025

Uh oh!

Uh oh!

Uh oh!

Fix interleaved_sliding_window config #8

Fix interleaved_sliding_window config #8

Uh oh!

Conversation

amogkam commented Jun 16, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Essential Elements of an Effective PR Description Checklist

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Uh oh!

github-actions bot commented Jun 16, 2025

Uh oh!

Uh oh!

Uh oh!

amogkam commented Jun 16, 2025 •

edited by github-actions bot

Loading