Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance lora tests with more layer and rank variations #3243

Merged
merged 18 commits into from
Mar 10, 2024
Merged

Enhance lora tests with more layer and rank variations #3243

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
221aaee
enhance lora tests
tterrysun Mar 6, 2024
d0cc1c9
add test fixture
tterrysun Mar 6, 2024
ccb1872
update requirements
tterrysun Mar 6, 2024
b35b7f1
remove redundant tests
tterrysun Mar 6, 2024
02ec9f9
minor fix
tterrysun Mar 7, 2024
2c820da
verify all tokens and refactor
tterrysun Mar 7, 2024
997f6a5
minorfix
tterrysun Mar 7, 2024
0baf6ba
minor fix
tterrysun Mar 7, 2024
cd2211e
check 32 tokens
tterrysun Mar 7, 2024
fa1f6f0
minor fix
tterrysun Mar 7, 2024
c80ad53
minor refactor
tterrysun Mar 7, 2024
991c577
Merge branch 'main' into lora_test_enhancement
tterrysun Mar 7, 2024
2fda771
minor fix
tterrysun Mar 7, 2024
1e48f26
minor fixes
tterrysun Mar 7, 2024
b430726
ablation study on ci
tterrysun Mar 8, 2024
4543e8f
use smaller model
tterrysun Mar 9, 2024
ce3dd58
add doc string
tterrysun Mar 9, 2024
0033822
remove redundant code
tterrysun Mar 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
remove redundant code
  • Loading branch information
@tterrysun
tterrysun committed Mar 9, 2024
commit 00338220c7e98d86bda928137f1e1b4c9c35fd60
4 changes: 0 additions & 4 deletions tests/lora/test_layer_variation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import pytest
from random import sample
import tempfile
import torch
from transformers import AutoModelForCausalLM

import vllm
Expand Down Expand Up @@ -73,9 +72,6 @@ def do_sample(llm,
@pytest.mark.parametrize("target_modules", TARGET_MODULES_LIST)
@pytest.mark.parametrize("rank", [8, 16, 32, 64])
def test_layer_variation_correctness(tp_size, target_modules, rank):
if torch.cuda.device_count() < tp_size:
pytest.skip(f"Not enough GPUs for tensor parallelism {tp_size}")

llm = vllm.LLM(MODEL_PATH,
enable_lora=True,
max_num_seqs=16,
Expand Down
Loading