Skip to content

Commit 9c8e2d1

Browse files
authored
[Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566)
1 parent 21313e0 commit 9c8e2d1

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/entrypoints/openai/api_server.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ def model_is_embedding(model_name: str, trust_remote_code: bool) -> bool:
6666
tokenizer_mode="auto",
6767
trust_remote_code=trust_remote_code,
6868
seed=0,
69-
dtype="float16").embedding_mode
69+
dtype="auto").embedding_mode
7070

7171

7272
@asynccontextmanager

0 commit comments

Comments
 (0)