Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model][VLM] Add LLaVA-Onevision model support #8486

Merged
merged 20 commits into from
Sep 22, 2024
Merged

[Model][VLM] Add LLaVA-Onevision model support #8486

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7b12c7f
add llava-onevision
Sep 12, 2024
159efb4
format: yapf and ruff
Sep 14, 2024
95f2dd0
update model notes
Sep 15, 2024
9256f17
format codes
Sep 15, 2024
4faa3af
add note for LLaVA-Onevision
litianjian Sep 15, 2024
705ea74
remove the tracing codes
Sep 15, 2024
a76aaee
Merge branch 'llava-one-vision' of https://github.com/litianjian/vllm…
Sep 15, 2024
ac3686a
Merge remote-tracking branch 'upstream/main' into llava-one-vision
ywang96 Sep 18, 2024
e8c820e
update example
ywang96 Sep 19, 2024
4f001dc
updateunpadded_image_feature_size function in llava_onevision
Sep 19, 2024
92c827f
update
ywang96 Sep 20, 2024
c511c73
Merge remote-tracking branch 'upstream/main' into llava-one-vision
ywang96 Sep 21, 2024
b8e5416
Merge remote-tracking branch 'upstream/main' into llava-one-vision
ywang96 Sep 22, 2024
f016735
Merge branch 'main' into llava-one-vision
DarkLight1337 Sep 22, 2024
c3adcd6
Fix registry test
DarkLight1337 Sep 22, 2024
9eaaea6
Apply #8656
DarkLight1337 Sep 22, 2024
2c8bad3
Fix dtype conversion error in LLaVA-OneVision test
DarkLight1337 Sep 22, 2024
23af922
Fix type annotation
DarkLight1337 Sep 22, 2024
8f8daa1
Don't remove the first token
DarkLight1337 Sep 22, 2024
f5cdd45
format
DarkLight1337 Sep 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Don't remove the first token
  • Loading branch information
@DarkLight1337
DarkLight1337 committed Sep 22, 2024
commit 8f8daa1c2253b7c8a04a67100b4f135d408f3e36
2 changes: 1 addition & 1 deletion tests/models/decoder_only/vision_language/test_llava_onevision.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def vllm_to_hf_output(vllm_output: Tuple[List[int], str,
if token_id != video_token_id or output_ids[idx - 1] != video_token_id
]

hf_output_str = output_str[1:]
hf_output_str = output_str
if hf_output_ids[-1] == eos_token_id:
hf_output_str = hf_output_str + tokenizer.decode(eos_token_id)

Expand Down
Loading