Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[misc][plugin] add plugin system implementation #7426

Merged
merged 40 commits into from
Aug 13, 2024
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7d25c1a
add plugin via entrypoints
youkaichao Aug 11, 2024
dc33035
add
youkaichao Aug 11, 2024
239e96e
use
youkaichao Aug 11, 2024
be7df65
update
youkaichao Aug 12, 2024
0632fb7
add model
youkaichao Aug 12, 2024
a2e1280
update
youkaichao Aug 12, 2024
a936ebc
use name
youkaichao Aug 12, 2024
5266296
update
youkaichao Aug 12, 2024
efd04f1
update tests
youkaichao Aug 12, 2024
06dddb7
update
youkaichao Aug 12, 2024
01248da
update
youkaichao Aug 12, 2024
3b2adc8
update
youkaichao Aug 12, 2024
93cb2cf
update
youkaichao Aug 12, 2024
b66400a
update
youkaichao Aug 12, 2024
a20a5bf
update server oot test
youkaichao Aug 12, 2024
8b9af15
update dummy args
youkaichao Aug 12, 2024
a560633
re-loadable
youkaichao Aug 12, 2024
eeff503
Merge branch 'main' into entrypoint_plugin
youkaichao Aug 12, 2024
e197d35
skip spell check for dummy model
youkaichao Aug 12, 2024
0074563
Merge branch 'entrypoint_plugin' of github.com:youkaichao/vllm into e…
youkaichao Aug 12, 2024
bd828b7
add vllm/plugins
youkaichao Aug 13, 2024
0b181e4
Merge branch 'main' into entrypoint_plugin
youkaichao Aug 13, 2024
6ba403c
Merge branch 'main' into entrypoint_plugin
youkaichao Aug 13, 2024
ae533fe
add files
youkaichao Aug 13, 2024
d69641a
lint
youkaichao Aug 13, 2024
aa57613
update
youkaichao Aug 13, 2024
5b5b06a
download script every time
youkaichao Aug 13, 2024
63193e8
remove model files
youkaichao Aug 13, 2024
7f64ac2
avoid too many downloads
youkaichao Aug 13, 2024
4457c32
Merge branch 'main' into entrypoint_plugin
youkaichao Aug 13, 2024
f9cd434
lint
youkaichao Aug 13, 2024
b03ab18
change to VLLM_PLUGINS
youkaichao Aug 13, 2024
d524219
revert changes
youkaichao Aug 13, 2024
2f07344
add tests
youkaichao Aug 13, 2024
f8797e5
add distributed tests
youkaichao Aug 13, 2024
b3099c2
add distributed tests
youkaichao Aug 13, 2024
3ad4c8e
add distributed tests
youkaichao Aug 13, 2024
18d2724
Merge branch 'main' into entrypoint_plugin
youkaichao Aug 13, 2024
27b77d8
add distributed tests
youkaichao Aug 13, 2024
e9471d2
Merge branch 'entrypoint_plugin' of github.com:youkaichao/vllm into e…
youkaichao Aug 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
download script every time
  • Loading branch information
youkaichao committed Aug 13, 2024
commit 5b5b06a02957aed52bdc58112502b8fcc48b144c
16 changes: 14 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import contextlib
import gc
import json
import os
import sys
from collections import UserList
Expand All @@ -11,6 +12,7 @@
import torch
import torch.nn as nn
import torch.nn.functional as F
from huggingface_hub import snapshot_download
from PIL import Image
from transformers import (AutoModelForCausalLM, AutoModelForSeq2SeqLM,
AutoModelForVision2Seq, AutoTokenizer, BatchEncoding,
Expand Down Expand Up @@ -761,5 +763,15 @@ def num_gpus_available():

@pytest.fixture
def dummy_opt_path():
cur_dir = os.path.dirname(__file__)
return os.path.join(cur_dir, "dummy_opt")
opt_path = snapshot_download(repo_id="facebook/opt-125m",
ignore_patterns=[
"*.bin", "*.bin.index.json", "*.pt",
"*.h5", "*.msgpack"
])
json_path = os.path.join(opt_path, "config.json")
with open(json_path, "r") as f:
config = json.load(f)
config["architectures"] = ["MyOPTForCausalLM"]
with open(json_path, "w") as f:
json.dump(config, f)
return opt_path