vLLM engine generates all zero tokens

[shaowei-su]

Hey there, I tried to launch MiniMaxAI/MiniMax-M1-40k with 8 H200 + vLLM (0.9.0).
The model generation outputs are always zeros:

'generated_tokens': [60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Is this a common issue with older version of vLLM?

[qscqesze]

Hi, thanks for the feedback!

I wasn’t able to reproduce this issue on my end. However, we recently pushed a fix related to the lm_head implementation, which could be relevant to the behavior you're seeing. lm_head issue

However, the Docker image for this vLLM update has not been released yet, so you'll need to rebuild vLLM locally yourself.

[shaowei-su]

Ah yes I think its related, I was not configuring the dtype explicitly to bf16 in my test and then it run into this issue. Closing this issue since you have solution put in already. Thanks