Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Upstream sync 2024 06 23 #329

Merged
merged 119 commits into from
Jun 26, 2024
Merged

Upstream sync 2024 06 23 #329

merged 119 commits into from
Jun 26, 2024

Conversation

robertgshaw2-neuralmagic
Copy link
Collaborator

@robertgshaw2-neuralmagic robertgshaw2-neuralmagic commented Jun 23, 2024 •
edited
Loading

SUMMARY:

Upstream sync 2024 06 23 (#329) - not a release candidare

SUMMARY:

Note that vllm-project@0f0d8bc is NOT included in this merge.

COMPARE vs UPSTREAM:

https://github.com/neuralmagic/nm-vllm/compare/upstream-sync-2024-06-23..6c916ac8a80d1b2f4e0d0113a67767dc254a3598

KuntaiDu and others added 30 commits June 23, 2024 21:18
@KuntaiDu @simon-mo @robertgshaw2-neuralmagic
[CI/Build][Misc] Add CI that benchmarks vllm performance on those PRs…
5d52fa5 
… with `perf-benchmarks` label (vllm-project#5073)

Co-authored-by: simon-mo <simon.mo@hey.com>
@DarkLight1337 @robertgshaw2-neuralmagic
[CI/Build] Disable LLaVA-NeXT CPU test (vllm-project#5529)
cab4a5d
@tlrmchlsmth @robertgshaw2-neuralmagic
[Kernel] Fix CUTLASS 3.x custom broadcast load epilogue (vllm-project…
923d05a 
…#5516)
@AllenDou @robertgshaw2-neuralmagic
[Misc] Fix arg names (vllm-project#5524)
34467ee
@robertgshaw2-neuralmagic @mgoin @dsikka
[ Misc ] Rs/compressed tensors cleanup (vllm-project#5432)
deee747 
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
@tlrmchlsmth @robertgshaw2-neuralmagic
[Kernel] Suppress mma.sp warning on CUDA 12.5 and later (vllm-project…
0ccb117 
…#5401)
@youkaichao @robertgshaw2-neuralmagic
[mis] fix flaky test of test_cuda_device_count_stateless (vllm-projec…
4464401 
…t#5546)
@DarkLight1337 @robertgshaw2-neuralmagic
[Core] Remove duplicate processing in async engine (vllm-project#5525)
28d0d6d
@youkaichao @robertgshaw2-neuralmagic
[misc][distributed] fix benign error in is_in_the_same_node (vllm-p…
f0e02ac 
…roject#5512)
@simon-mo @robertgshaw2-neuralmagic
[Docs] Add ZhenFund as a Sponsor (vllm-project#5548)
d0a3026
@sangstar @robertgshaw2-neuralmagic
[Doc] Update documentation on Tensorizer (vllm-project#5471)
33edc9b
@tdoublep @robertgshaw2-neuralmagic
[Bugfix] Enable loading FP8 checkpoints for gpt_bigcode models (vllm-…
5fffeb8 
…project#5460)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@WoosukKwon @robertgshaw2-neuralmagic
[Bugfix] Fix typo in Pallas backend (vllm-project#5558)
65419f4
@youkaichao @robertgshaw2-neuralmagic
[Core][Distributed] improve p2p cache generation (vllm-project#5528)
dfd2b2e
@simon-mo @robertgshaw2-neuralmagic
Add ccache to amd (vllm-project#5555)
d464106
@leiwen83 @wenlei03 @robertgshaw2-neuralmagic
[Core][Bugfix]: fix prefix caching for blockv2 (vllm-project#5364)
80b908f 
Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
@DarkLight1337 @robertgshaw2-neuralmagic
[mypy] Enable type checking for test directory (vllm-project#5017)
0393d45
@DarkLight1337 @robertgshaw2-neuralmagic
[CI/Build] Test both text and token IDs in batched OpenAI Completions…
32d5ecc 
… API (vllm-project#5568)
@rkooo567 @DarkLight1337 @robertgshaw2-neuralmagic
[misc] Do not allow to use lora with chunked prefill. (vllm-project#5538
6f3169a 
)

Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@alexm-neuralmagic @robertgshaw2-neuralmagic
add gptq_marlin test for bug report vllm-project#5088 (vllm-project#5145
beb3b21 
)
@njhill @robertgshaw2-neuralmagic
[BugFix] Don't start a Ray cluster when not using Ray (vllm-project#5570
31f38f3 
)
@zifeitong @robertgshaw2-neuralmagic
[Fix] Correct OpenAI batch response format (vllm-project#5554)
ec68cd1
@Yard1 @robertgshaw2-neuralmagic
Add basic correctness 2 GPU tests to 4 GPU pipeline (vllm-project#5518)
dc8789d
@mgoin @robertgshaw2-neuralmagic
[CI][BugFix] Flip is_quant_method_supported condition (vllm-project#5577
681de21 
)
@youkaichao @robertgshaw2-neuralmagic
[build][misc] limit numpy version (vllm-project#5582)
77a5f36
@youkaichao @robertgshaw2-neuralmagic
[Doc] add debugging tips for crash and multi-node debugging (vllm-pro…
9c77244 
…ject#5581)
@comaniac @robertgshaw2-neuralmagic
Fix w8a8 benchmark and add Llama-3-8B (vllm-project#5562)
f968328
@garg-amit @robertgshaw2-neuralmagic
[Model] Rename Phi3 rope scaling type (vllm-project#5595)
b0abad9
@CharlesRiggins @robertgshaw2-neuralmagic
Correct alignment in the seq_len diagram. (vllm-project#5592)
4b84959 
Co-authored-by: Liqian Chen <liqian.chen@deeplang.ai>
@dsikka @robertgshaw2-neuralmagic
[Kernel] compressed-tensors marlin 24 support (vllm-project#5435)
9cfb1d7
WoosukKwon and others added 26 commits June 23, 2024 21:24
@WoosukKwon @robertgshaw2-neuralmagic
[Docs][TPU] Add installation tip for TPU (vllm-project#5761)
dc211cd
@youkaichao @robertgshaw2-neuralmagic
[core][distributed] improve shared memory broadcast (vllm-project#5754)
860a1d6
@varun-sundar-rabindranath @robertgshaw2-neuralmagic
[BugFix] [Kernel] Add Cutlass2x fallback kernels (vllm-project#5744)
d7f0ece 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
@robertgshaw2-neuralmagic
remove vllm-runnner-nm
e484da4
@robertgshaw2-neuralmagic
formatted
683f309
@robertgshaw2-neuralmagic
fix is_xpu
3b3a92c
@robertgshaw2-neuralmagic
fix lm eval
e0c0530
@robertgshaw2-neuralmagic
fix format
01d4f34
@robertgshaw2-neuralmagic
Merge branch 'main' into upstream-sync-2024-06-23
616fce8
@robertgshaw2-neuralmagic
remove flaky gptq models
3c5a7f5
@robertgshaw2-neuralmagic
fix import error
71f60a8
@robertgshaw2-neuralmagic
format
e960ebb
@robertgshaw2-neuralmagic
fix lm-eval
3297247
@robertgshaw2-neuralmagic
Merge branch 'main' into upstream-sync-2024-06-23
dcdf4da
@robertgshaw2-neuralmagic
format
0dd1848
@robertgshaw2-neuralmagic
Merge branch 'upstream-sync-2024-06-23' of https://github.com/neuralm…
de06faa 
…agic/nm-vllm into upstream-sync-2024-06-23
@robertgshaw2-neuralmagic
fix test tracing
cdf52bf
@robertgshaw2-neuralmagic
Merge branch 'main' into upstream-sync-2024-06-23
f2d2794
@robertgshaw2-neuralmagic
fixed test tracing deps
431054d
@robertgshaw2-neuralmagic
added skipping logic
9d7b7b5
@robertgshaw2-neuralmagic
handled skipping
3a75e15
@robertgshaw2-neuralmagic
updated to run only on solo
c9d1b9e
@robertgshaw2-neuralmagic
format
973d9d0
@robertgshaw2-neuralmagic
clean up
c44802e
@robertgshaw2-neuralmagic
cleanup
9c15fe1
@robertgshaw2-neuralmagic
format
727077f
mgoin
mgoin approved these changes Jun 26, 2024
View reviewed changes
Copy link
Member

@mgoin mgoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked carefully through and LGTM!

@robertgshaw2-neuralmagic robertgshaw2-neuralmagic merged commit b4ad97a into main Jun 26, 2024
48 of 53 checks passed
@robertgshaw2-neuralmagic robertgshaw2-neuralmagic deleted the upstream-sync-2024-06-23 branch June 26, 2024 14:37
@robertgshaw2-neuralmagic robertgshaw2-neuralmagic mentioned this pull request Jul 1, 2024
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mgoin mgoin mgoin approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.