Fix hanging when prompt exceeds limit

[chenxu2048]

I discovered that my API server with AsyncLLMEngine was blocked when a lengthy prompt was submitted to it. This problem shares similarities with issues #765 and #1026.

The scheduler is adding seq_groups that are larger than the prompt_limit:

if num_prompt_tokens > self.prompt_limit:
    logger.warning(
        f"Input prompt ({num_prompt_tokens} tokens) is too long"
        f" and exceeds limit of {self.prompt_limit}")
    for seq in seq_group.get_seqs():
        seq.status = SequenceStatus.FINISHED_IGNORED
    ignored_seq_groups.append(seq_group)
    self.waiting.pop(0)
    continue

And it returns when the schedule is not empty:

if scheduled:
    scheduler_outputs = SchedulerOutputs(
        scheduled_seq_groups=scheduled,
        prompt_run=True,
        num_batched_tokens=num_batched_tokens,
        blocks_to_swap_in=blocks_to_swap_in,
        blocks_to_swap_out=blocks_to_swap_out,
        blocks_to_copy=blocks_to_copy,
        ignored_seq_groups=ignored_seq_groups,
    )
    return scheduler_outputs

However, in LLMEngine._schedule, ignored_seq_groups is returned when scheduler_outputs.is_empty().

def step(self) -> List[RequestOutput]:
    (seq_group_metadata_list, scheduler_outputs,
     early_return) = self._schedule()
    if early_return is not None:
        return early_return

The third value returned by LLMEngine._schedule will always be None, and the ignored seq groups are thus LEAKED.

[zhuohan123]

LGTM! Thanks for the great fix!