Skip to content

Commit

Permalink
[V1] Bugfix: Validate Model Input Length (vllm-project#12600)
Browse files Browse the repository at this point in the history
SUMMARY:
* avoid crashing the engine when we get an input longer than
max_model_len

FIX vllm-project#12567(*link existing issues this PR will resolve*)
  • Loading branch information
robertgshaw2-redhat authored Feb 1, 2025
1 parent 44bbca7 commit b1340f9
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions vllm/v1/engine/processor.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,11 @@ def _validate_model_inputs(self, inputs: ProcessorInputs):
if prompt_ids is None or len(prompt_ids) == 0:
raise ValueError("Prompt cannot be empty")

if len(prompt_ids) >= self.model_config.max_model_len:
raise ValueError(
f"Prompt length of {len(prompt_ids)} is longer than the "
f"maximum model length of {self.model_config.max_model_len}.")

if self.model_config.is_multimodal_model:
max_prompt_len = self.model_config.max_model_len

Expand Down

0 comments on commit b1340f9

Please sign in to comment.