Skip to content

Commit 128425b

Browse files
joerundeAlvant
authored andcommitted
[Bugfix] Fix TP > 1 for new granite (vllm-project#8544)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com> Signed-off-by: Alvant <alvasian@yandex.ru>
1 parent d5f7c8b commit 128425b

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/model_executor/models/granite.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -428,7 +428,8 @@ def compute_logits(
428428
sampling_metadata: SamplingMetadata) -> Optional[torch.Tensor]:
429429
logits = self.logits_processor(self.lm_head, hidden_states,
430430
sampling_metadata)
431-
logits /= self.config.logits_scaling
431+
if logits is not None:
432+
logits /= self.config.logits_scaling
432433
return logits
433434

434435
def sample(

0 commit comments

Comments
 (0)