Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Upstream sync 2024 03 18 #134

Merged
merged 133 commits into from
Mar 19, 2024
Merged

Upstream sync 2024 03 18 #134

merged 133 commits into from
Mar 19, 2024

Conversation

robertgshaw2-neuralmagic
Copy link
Collaborator

@robertgshaw2-neuralmagic robertgshaw2-neuralmagic commented Mar 18, 2024 •
edited
Loading

SUMMARY:

  • upstream merge (sync) up to 93348d9458af7517bb8c114611d438a1b4a2c3be
  • some minor changes related to ruff and yapf

NOTES: skipped flaky lora gemma test

TEST PLAN:
ran nightly, passed all except gemma
running now on remote push

ronensc and others added 30 commits February 21, 2024 18:18
@ronensc
Update comment (vllm-project#2934)
d7f3964
@Maxusmusti
Added early stopping to completion APIs (vllm-project#2939)
5574081
@esmeetu
Migrate MistralForCausalLM to LlamaForCausalLM (vllm-project#2868)
344020c
@WoosukKwon
Use Llama RMSNorm custom op for Gemma (vllm-project#2974)
95529e3
@mspronesti
chore(vllm): codespell for spell checking (vllm-project#2820)
93dc5a2
@WoosukKwon
Optimize GeGLU layer in Gemma (vllm-project#2975)
fd5dcc5
@44670
[FIX] Fix a bug in initializing Yarn RoPE (vllm-project#2983)
c530e2c
@WoosukKwon
Remove Flash Attention in test env (vllm-project#2982)
6f32cdd
@ronensc
Include tokens from prompt phase in counter_generation_tokens (vllm…
4caf704 
…-project#2802)
@zhaoyang-star
Fix nvcc not found in vlm-openai image (vllm-project#2781)
57f0449
@WoosukKwon
[Fix] Fissertion on YaRN model len (vllm-project#2984)
f7c1234
@hmellor
Port metrics from aioprometheus to prometheus_client (vllm-projec…
ef978fe 
…t#2730)
@jlcmoore
Add LogProbs for Chat Completions in OpenAI (vllm-project#2918)
70f3e8e
@pcmoritz @cadedaniel
Optimize Triton MoE Kernel (vllm-project#2979)
cfc15a1 
Co-authored-by: Cade Daniel <edacih@gmail.com>
@WoosukKwon
[Minor] Remove gather_cached_kv kernel (vllm-project#3043)
d6e4a13
@esmeetu
[Minor] Remove unused config files (vllm-project#3039)
d9f726c
@esmeetu
Don't use cupy when enforce_eager=True (vllm-project#3037)
c1c0d00
@esmeetu
Fix stablelm (vllm-project#3038)
4dd6416
@dachengai @WoosukKwon
Support Orion model (vllm-project#2539)
48a8f4a 
Co-authored-by: zhangdacheng <zhangdacheng@ainirobot.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@Jingru
fix get_ip error in pure ipv6 environment (vllm-project#2931)
2410e32
@WoosukKwon
[Minor] Fix type annotation in fused moe (vllm-project#3045)
4bd18ec
@dylanwhawk
Support logit bias for OpenAI API (vllm-project#3027)
e0ade06
@WoosukKwon
[Minor] Fix StableLMEpochForCausalLM -> StableLmForCausalLM (vllm-pro…
8b430d7 
…ject#3046)
@sighingnow
Enable GQA support in the prefix prefill kernels (vllm-project#3007)
71bcaf9 
Signed-off-by: Tao He <sighingnow@gmail.com>
@ElefHead
multi-lora documentation fix (vllm-project#3064)
a868310
@AllenDou
Restrict prometheus_client >= 0.18.0 to prevent errors when importing…
e46fa5d 
… pkgs (vllm-project#3070)
@liangfu
[Neuron] Support inference with transformers-neuronx (vllm-project#2569)
3b7178c
@WoosukKwon
Add LoRA support for Gemma (vllm-project#3050)
929b4f2
@chu-tianxiang
Add Support for 2/3/8-bit GPTQ Quantization Models (vllm-project#2330)
01a5d18
@jaywonchung
Fix: AttributeError in OpenAI-compatible server (vllm-project#3018)
a6d471c
declark1 and others added 24 commits March 15, 2024 09:56
@declark1
[Fix] Add args for mTLS support (vllm-project#3430)
03d37f2 
Co-authored-by: declark1 <daniel.clark@ibm.com>
@sighingnow @simon-mo
Fixes the misuse/mixuse of time.time()/time.monotonic() (vllm-project…
14b8ae0 
…#3220)

Signed-off-by: Tao He <sighingnow@gmail.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
@youkaichao
[Misc] add error message in non linux platform (vllm-project#3438)
604f235
@hmellor
Fix issue templates (vllm-project#3436)
a7af453
@laneeeee
fix document error for value and v_vec illustration (vllm-project#3421)
8fa7357
@Yard1
Asynchronous tokenization (vllm-project#2879)
fb96c1e
@robertgshaw2-neuralmagic
Removed Extraneous Print Message From OAI Server (vllm-project#3440)
10585e0
@youkaichao @zhuohan123
[Misc] PR templates (vllm-project#3413)
413366e 
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
@sighingnow
Fixes the incorrect argument in the prefix-prefill test cases (vllm-p…
3123f15 
…roject#3246)
@ronensc
Replace lstrip() with removeprefix() to fix Ruff linter warning (v…
14e3f9a 
…llm-project#2958)
@Dinghow
Fix Baichuan chat template (vllm-project#3340)
cf6ff18
@simon-mo
fix lint
ad50bf4
@simon-mo
[Misc] fix line length for entire codebase (vllm-project#3444)
8e67598
@simon-mo
Support arbitrary json_object in OpenAI and Context Free Grammar (vll…
120157f 
…m-project#3211)
@simon-mo
Fix setup.py neuron-ls issue (vllm-project#2671)
6b78837
@WoosukKwon @youkaichao
[Misc] Use dataclass for InputMetadata (vllm-project#3452)
abfc4f3 
Co-authored-by: youkaichao <youkaichao@126.com>
@simon-mo
[CI] Shard tests for LoRA and Kernels to speed up (vllm-project#3445)
93348d9
@robertgshaw2-neuralmagic
Merge remote-tracking branch 'origin/upstream-main' into upstream-syn…
79d7994 
…c-2024-03-18
@robertgshaw2-neuralmagic
fixed bad merge
5ef5f3c
@robertgshaw2-neuralmagic
format
a856145
@robertgshaw2-neuralmagic
fixed failed test
7d913f8
@robertgshaw2-neuralmagic
skipped flaky gemma lora test for automation
c6b2f62
@robertgshaw2-neuralmagic
format
68e4239
@robertgshaw2-neuralmagic
Merge branch 'main' into upstream-sync-2024-03-18
793abab
mgoin
mgoin approved these changes Mar 19, 2024
View reviewed changes
Copy link
Member

@mgoin mgoin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, love the "UPSTREAM SYNC"s and just one question

tests/test_cache_block_hashing.py Show resolved Hide resolved
@robertgshaw2-neuralmagic robertgshaw2-neuralmagic merged commit 7ab58f7 into main Mar 19, 2024
2 checks passed
@robertgshaw2-neuralmagic robertgshaw2-neuralmagic deleted the upstream-sync-2024-03-18 branch March 19, 2024 19:33
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mgoin mgoin mgoin approved these changes

@varun-sundar-rabindranath varun-sundar-rabindranath Awaiting requested review from varun-sundar-rabindranath

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.