[Feature]: Add prompt token ids into logit processor?

[xz-liu]

🚀 The feature, motivation and pitch

I am using vllm's SamplingParams to set up the logit processor. It looks the same as huggingface, but there are some subtle differences in use. The main difference is that it does not give the logit processor all the token IDs, but starts from the generated token.

On the contrary, huggingface provides the prompt and the token ID of the generated token. I was wondering if there is a way to allow the use of vllm's logit processor to also obtain this information?

Alternatives

No response

Additional context

No response

[simon-mo]

good point. i think this should be added so contribution welcomed. curious about the use case though, what are some intended use of the prompt tokens for logits processors?

[kezouke]

curious

An example where it is necessary to get prompt tokens ids:

The only way to get logits for all tokens from vocabulary is to use the logit processor function.

In a constrained generation setting, you need to store the logits for a specific set of tokens to determine which one is most probable within the context provided by the language model. This set of tokens, whose logits are compared, changes each time llm.generate() is called and depends on the prompt given to the model.

[DarkLight1337]

Fixed by #4985