Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Upstream sync 2024 04 08 #173

Merged
merged 292 commits into from
Apr 10, 2024
Merged

Upstream sync 2024 04 08 #173

merged 292 commits into from
Apr 10, 2024

Conversation

SageMoore
Copy link

No description provided.

cloudhan and others added 30 commits March 2, 2024 14:34
@cloudhan
Reorder kv dtype check to avoid nvcc not found error on AMD platform (v…
baee28c 
…llm-project#3104)
@SageMoore @ElizaWszola @mgoin
Add Automatic Prefix Caching (vllm-project#2762)
ce4f5a2 
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
@jasonacox
Add vLLM version info to logs and openai API server (vllm-project#3161)
d65fac2
@zhuohan123
[FIX] Fix styles in automatic prefix caching & add a automatic prefix…
996d095 
… caching benchmark (vllm-project#3158)
@pcmoritz @ywang96
Make it easy to profile workers with nsight (vllm-project#3162)
17c3103 
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
@liangfu
[DOC] add setup document to support neuron backend (vllm-project#2777)
d0fae88
@gty111
[Minor Fix] Remove unused code in benchmark_prefix_caching.py (vllm-p…
901cf4c 
…roject#3171)
@pian13131
Add document for vllm paged attention kernel. (vllm-project#2978)
27a7b07
@AllenDou
enable --gpu-memory-utilization in benchmark_throughput.py (vllm-proj…
9cbc7e5 
…ect#3175)

Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
@ttbachyinsda
[Minor fix] The domain dns.google may cause a socket.gaierror excepti…
76e8a70 
…on (vllm-project#3176)

Co-authored-by: guofangze <guofangze@kuaishou.com>
@Yard1 @avnishn
Push logprob generation to LLMEngine (vllm-project#3065)
22de452 
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
@Yard1 @zhuohan123
Add health check, make async Engine more robust (vllm-project#3015)
ff578ca 
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
@wangchen615 @ywang96
Fix the openai benchmarking requests to work with latest OpenAI apis (v…
9a4548b 
…llm-project#2992)

Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
@hongxiayang @lcskrishna
[ROCm] enable cupy in order to enable cudagraph mode for AMD GPUs (vl…
05af6da 
…lm-project#3123)

Co-authored-by: lcskrishna <lollachaitanya@gmail.com>
@njhill
Store eos_token_id in Sequence for easy access (vllm-project#3166)
8999ec3
@njhill @Yard1
[Fix] Avoid pickling entire LLMEngine for Ray workers (vllm-project#3207
2efce05 
)

Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
@rkooo567
[Tests] Add block manager and scheduler tests (vllm-project#3108)
24aecf4
@cadedaniel
[Testing] Fix core tests (vllm-project#3224)
a33ce60
@chujiezheng
Add tqdm dynamic_ncols=True (vllm-project#3242)
4cb3b92
@TechxGenus
Add GPTQ support for Gemma (vllm-project#3200)
d3c04b6
@wangchen615 @zhuohan123
Update requirements-dev.txt to include package for benchmarking scrip…
cbf4c05 
…ts. (vllm-project#3181)

Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
@WoosukKwon
Separate attention backends (vllm-project#3005)
2daf23a
@mgoin
Measure model memory usage (vllm-project#3120)
385da2d
@jacobthebanana
Possible fix for conflict between Automated Prefix Caching (vllm-proj…
8cbba46 
…ect#2762) and multi-LoRA support (vllm-project#1804) (vllm-project#3263)
@ElizaWszola
Fix auto prefix bug (vllm-project#3239)
b35cc93
@njhill
Connect engine healthcheck to openai server (vllm-project#3260)
d2339d6
@whyiug
Feature add lora support for Qwen2 (vllm-project#3177)
c59e120
@gty111
[Minor Fix] Fix comments in benchmark_serving (vllm-project#3252)
1ece1ae
@ywang96
[Docs] Fix Unmocked Imports (vllm-project#3275)
99c3cfb
@WoosukKwon
[FIX] Make flash_attn optional (vllm-project#3269)
1cb0cc2
cadedaniel and others added 13 commits April 4, 2024 21:54
@cadedaniel
[Misc] Add pytest marker to opt-out of global test cleanup (vllm-proj…
e5043a3 
…ect#3863)
@cadedaniel
[Misc] Fix linter issues in examples/fp8/quantizer/quantize.py (vllm-…
e0dd4d3 
…project#3864)
@noamgat
[Bugfix] Fixing requirements.txt (vllm-project#3865)
9edec65
@WoosukKwon
[Misc] Define common requirements (vllm-project#3841)
cfaf49a
@tdoublep
Add option to completion API to truncate prompt tokens (vllm-project#…
1d7c940 
…3144)
@rkooo567
[Chunked Prefill][4/n] Chunked prefill scheduler. (vllm-project#3853)
18de883
@Isotr0py
[Bugfix] Fix incorrect output on OLMo models in Tensor Parallelism (v…
54951ac 
…llm-project#3869)
@youkaichao
[CI/Benchmark] add more iteration and use median for robust latency b…
e4be7d7 
…enchmark (vllm-project#3889)
@youkaichao
[Core] enable out-of-tree model register (vllm-project#3871)
95baec8
@youkaichao
[Core] latency optimization (vllm-project#3890)
2f19283
@Isotr0py
[Bugfix] Fix Llava inference with Tensor Parallelism. (vllm-project#3883
0ce0539 
)
@SUDA-HLT-ywfang
[Model] add minicpm (vllm-project#3893)
b4543c8
@SageMoore
init
52d61ba
mgoin
mgoin reviewed Apr 8, 2024
View reviewed changes
pyproject.toml Show resolved Hide resolved
SageMoore
SageMoore commented Apr 8, 2024
View reviewed changes
Dockerfile Outdated Show resolved Hide resolved
@mgoin mgoin mentioned this pull request Apr 8, 2024
Closed
mgoin
mgoin reviewed Apr 9, 2024
View reviewed changes
README.md Show resolved Hide resolved
andy-neuma
andy-neuma approved these changes Apr 9, 2024
View reviewed changes
Copy link
Member

@andy-neuma andy-neuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

.github/actions/nm-build-vllm/action.yml Outdated Show resolved Hide resolved
.github/scripts/build Outdated Show resolved Hide resolved
.github/workflows/publish.yml Show resolved Hide resolved
.github/workflows/scripts/build.sh Outdated Show resolved Hide resolved
Dockerfile
COPY csrc csrc
COPY setup.py setup.py
COPY cmake cmake
COPY CMakeLists.txt CMakeLists.txt
COPY requirements.txt requirements.txt
COPY requirements-common.txt requirements-common.txt
COPY requirements-cuda.txt requirements-cuda.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need "requirements-dev.txt" or is this file being deprecated?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. I'll check to see if it looks like we need anything in there.

Dockerfile Show resolved Hide resolved
@SageMoore SageMoore merged commit cb7bad6 into main Apr 10, 2024
2 checks passed
@SageMoore SageMoore deleted the upstream-sync-2024-04-08 branch April 10, 2024 17:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mgoin mgoin mgoin left review comments

@andy-neuma andy-neuma andy-neuma approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.