Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do not merge] ibm-dev build ibm-20241018 #209

Closed
wants to merge 56 commits into from
Closed

[do not merge] ibm-dev build ibm-20241018 #209

wants to merge 56 commits into from

Conversation

fialhocoelho
Copy link

PR to trigger the build process for a compiled image

WoosukKwon and others added 30 commits October 14, 2024 15:02
@WoosukKwon
[TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (vllm-project…
473e7b3 
…#9350)
@LunrEclipse
[Frontend] merge beam search implementations (vllm-project#9296)
4d31cd4
@xiangxu-google
[Model] Make llama3.2 support multiple and interleaved images (vllm-p…
f0fe4fe 
…roject#9095)
@tlrmchlsmth
[Bugfix] Clean up some cruft in mamba.py (vllm-project#9343)
169b530
@stevegrubb
[Frontend] Clarify model_type error messages (vllm-project#9345)
44eaa5a
@mgoin
[Doc] Fix code formatting in spec_decode.rst (vllm-project#9348)
8e836d9
@hhzhang16
[Bugfix] Update InternVL input mapper to support image embeds (vllm-p…
55e081f 
…roject#9351)
@njhill
[BugFix] Fix chat API continuous usage stats (vllm-project#9357)
e9d517f
@gracehonv
pass ignore_eos parameter to all benchmark_serving calls (vllm-projec…
5d264f4 
…t#9349)
@mgoin @DarkLight1337
[Misc] Directly use compressed-tensors for checkpoint definitions (vl…
22f8a69 
…lm-project#8909)

Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
@CatherineSue @njhill
[Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with emp…
ba30942 
…ty token_ids (vllm-project#9034)

Co-authored-by: Nick Hill <nickhill@us.ibm.com>
@LucasWilkinson
[Bugfix][CI/Build] Fix CUDA 11.8 Build (vllm-project#9386)
717a5f8
@mrsalehi @sangho-vision @ywang96
[Bugfix] Molmo text-only input bug fix (vllm-project#9397)
ed92013 
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
@DarkLight1337
[Misc] Standardize RoPE handling for Qwen2-VL (vllm-project#9250)
7e7eae3
@DarkLight1337
[Model] VLM2Vec, the first multimodal embedding model in vLLM (vllm-p…
7abba39 
…roject#9303)
@DarkLight1337
[CI/Build] Test VLM embeddings (vllm-project#9406)
1de76a0
@DarkLight1337
[Core] Rename input data types (vllm-project#8688)
cee711f
@ywang96 @DarkLight1337
[Misc] Consolidate example usage of OpenAI client for multimodal mode…
59230ef 
…ls (vllm-project#9412)

Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
@Isotr0py
[Model] Support SDPA attention for Molmo vision backbone (vllm-projec…
cf1d62a 
…t#9410)
@patrickvonplaten
Support mistral interleaved attn (vllm-project#9414)
415f76a
@mzusman
[Kernel][Model] Improve continuous batching for Jamba and Mamba (vllm…
fb60ae9 
…-project#9189)
@streaver91
[Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCP…
5b8a1fd 
…M-S-1B-sft (vllm-project#9396)
@LiuXiaoxuanPKU
[Performance][Spec Decode] Optimize ngram lookup performance (vllm-pr…
8345045 
…oject#9333)
@russellb
[CI/Build] mypy: Resolve some errors from checking vllm/engine (vllm-…
776dbd7 
…project#9267)

Signed-off-by: Russell Bryant <rbryant@redhat.com>
@tlrmchlsmth
[Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token qu…
c3fab5f 
…antize kernel (vllm-project#9425)
@rasmith
[BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (vllm-projec…
92d86da 
…t#9391)
@terrytangyuan
Add notes on the use of Slack (vllm-project#9442)
dbfa8d3
@LucasWilkinson
[Kernel] Add Exllama as a backend for compressed-tensors (vllm-projec…
e312e52 
…t#9395)
@DarkLight1337
[Misc] Print stack trace using logger.exception (vllm-project#9461)
390be74
@LucasWilkinson @mgoin
[misc] CUDA Time Layerwise Profiler (vllm-project#8337)
9d30a05 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
dtrifiro and others added 22 commits October 17, 2024 10:25
@dtrifiro
[CI/Build] remove .github from .dockerignore, add dirty repo check (v…
a2c71c5 
…llm-project#9375)
@DarkLight1337
[Misc] Remove commit id file (vllm-project#9470)
7871659
@ProExpertProg
[torch.compile] Fine-grained CustomOp enabling mechanism (vllm-projec…
0f41fbe 
…t#9300)
@bnellnm
[Bugfix] Fix support for dimension like integers and ScalarType (vllm…
eca2c5f 
…-project#9299)
@wukaixingxp @Isotr0py
[Bugfix] Add random_seed to sample_hf_requests in benchmark_serving s…
d65049d 
…cript (vllm-project#9013)

Co-authored-by: Isotr0py <2037008807@qq.com>
@sasha0552
[Bugfix] Print warnings related to mistral_common tokenizer only on…
d615b5c 
…ce (vllm-project#9468)
@sssrijan-amazon
[Hardwware][Neuron] Simplify model load for transformers-neuronx libr…
bb76538 
…ary (vllm-project#9380)
@robertgshaw2-neuralmagic @afeldman-nm
@WoosukKwon @laishzh @maxdebayser @DarkLight1337
Support BERTModel (first encoder-only embedding model) (vllm-proj…
343f8e0 
…ect#9056)

Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: laishzh <laishengzhang@gmail.com>
Co-authored-by: Max de Bayser <maxdebayser@gmail.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@dsikka
[BugFix] Stop silent failures on compressed-tensors parsing (vllm-pro…
48138a8 
…ject#9381)
@joerunde
[Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory u…
de4008e 
…sage (vllm-project#9352)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
@blueyo0
[Qwen2.5] Support bnb quant for Qwen2.5 (vllm-project#9467)
154a8ae
@russellb
[CI/Build] Use commit hash references for github actions (vllm-projec…
944dd8e 
…t#9430)
@njhill
[BugFix] Typing fixes to RequestOutput.prompt and beam search (vllm-p…
1ffc8a7 
…roject#9473)
@tomeras91
[Frontend][Feature] Add jamba tool parser (vllm-project#9154)
d2b1bf5
@njhill
[BugFix] Fix and simplify completion API usage streaming (vllm-projec…
25aeb7d 
…t#9475)
@DarkLight1337
[CI/Build] Fix lint errors in mistral tokenizer (vllm-project#9504)
1bbbcc0
@tlrmchlsmth
[Bugfix] Fix offline_inference_with_prefix.py (vllm-project#9505)
ae8b633
@fialhocoelho
Merge branch 'main' of https://github.com/vllm-project/vllm into ibm-…
5c074b3 
…20241018

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 5733
682e622 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 6357
a823383 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 9027
1105122 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 9387
05fe6cb 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 18, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: fialhocoelho
Once this PR has been reviewed and has the lgtm label, please assign vaibhavjainwiz for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 18, 2024

@fialhocoelho: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/smoke-test 05fe6cb link true /test smoke-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

prarit pushed a commit to prarit/vllm that referenced this pull request Oct 21, 2024
@sanyalington
re-enable avoid torch slice fix when chunked prefill is disabled (ope…
a5d87a1 
…ndatahub-io#209)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@heyselbi heyselbi Awaiting requested review from heyselbi

@Xaenalt Xaenalt Awaiting requested review from Xaenalt

Assignees
No one assigned
Labels
do-not-merge/hold
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.