Skip to content

Commit 85b1d42

Browse files
committed
fix guided decoding
Signed-off-by: MengqingCao <cmq0113@163.com>
1 parent 5ba2396 commit 85b1d42

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2000,7 +2000,7 @@ def execute_model(
20002000
logits = model_output_broadcast_data["logits"]
20012001

20022002
# Apply structured output bitmasks if present
2003-
if scheduler_output.grammar_bitmask is not None:
2003+
if scheduler_output.structured_output_request_ids:
20042004
logits = self.apply_grammar_bitmask(scheduler_output, logits)
20052005

20062006
# Sample the next token and get logprobs if needed.

0 commit comments

Comments
 (0)