[model] support vision language model llava. #178
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
liutongxuan
force-pushed
the
features/llava
branch
from
bfa4b3c to
45db749
Compare
liutongxuan
force-pushed
the
features/llava
branch
3 times, most recently
from
1fb4495 to
68013b4
Compare
liutongxuan
changed the title
liutongxuan
force-pushed
the
features/llava
branch
from
96fb59b to
0ea99b6
Compare
liutongxuan
force-pushed
the
features/llava
branch
10 times, most recently
from
7182aad to
4e99594
Compare
liutongxuan
force-pushed
the
features/llava
branch
6 times, most recently
from
3c41efc to
6dd9cea
Compare
guocuimi
reviewed
Jun 28, 2024
scalellm/_C/vlm_handler.pyi
Outdated
| from scalellm._C.output import RequestOutput | ||
| from scalellm._C.sampling_params import SamplingParams | ||
|
|
||
| class Priority(Enum): |
There was a problem hiding this comment.
redefinition error for Priority.
guocuimi
reviewed
Jun 28, 2024
| @@ -49,6 +49,8 @@ void Batch::add(Sequence* sequence, uint32_t token_budget) { | |||
| sequences_.push_back(sequence); | |||
| token_budgets_.push_back(token_budget); | |||
| budget_used_.push_back(0); | |||
|
|
|||
| input_embedding_ = sequence->get_input_embedding(); | |||
There was a problem hiding this comment.
sounds one input embedding for each sequence, if so, it should be a vector of input embeddings for the batch.
There was a problem hiding this comment.
yes, in this pr only support one sequence + one image, and will support multiple sequences later.
|
LGTM. Thank you for your hard work in getting the first VLM up and running. I've left some comments; I believe we should refactor to reduce redundant code between the worker and engine after landing this. |
liutongxuan
force-pushed
the
features/llava
branch
4 times, most recently
from
3daa679 to
137360a
Compare
liutongxuan
force-pushed
the
features/llava
branch
from
137360a to
baa26a4
Compare
guocuimi
added a commit
that referenced
this pull request
Jul 3, 2024
Let's revert the VLM related change for now to unblock release. cc: @liutongxuan - [ ] @guocuimi to create a branch for VLM development. This reverts commit 437be3f and d711c55.
guocuimi
pushed a commit
that referenced
this pull request
Jul 3, 2024
(cherry picked from commit 437be3f)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.