Skip to content

Commit 68af77e

Browse files
authored
[FIXBUG] Correctly Apply Grammar Bitmask in Mixed Batches (#22896)
Signed-off-by: JartX <sagformas@epdcenter.es>
1 parent 6b04039 commit 68af77e

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

vllm/v1/worker/gpu_model_runner.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1337,9 +1337,10 @@ def apply_grammar_bitmask(
13371337
out_indices = []
13381338

13391339
# Reorder the bitmask to match the order of the requests in the batch.
1340-
sorted_bitmask = np.zeros_like(grammar_bitmask,
1341-
shape=(logits.shape[0],
1342-
grammar_bitmask.shape[1]))
1340+
sorted_bitmask = np.full(shape=(logits.shape[0],
1341+
grammar_bitmask.shape[1]),
1342+
fill_value=-1,
1343+
dtype=grammar_bitmask.dtype)
13431344
cumulative_index = 0
13441345
seq = sorted(scheduler_output.structured_output_request_ids.items(),
13451346
key=lambda x: x[1])

0 commit comments

Comments
 (0)