Skip to content

Comments

dont request prompt logprobs as we dont use them and they disable prefix cache and burst vllm memory usage#666

Merged
Jackmin801 merged 1 commit intomainfrom
no-prompt-logprobs
Dec 27, 2025
Merged

dont request prompt logprobs as we dont use them and they disable prefix cache and burst vllm memory usage#666
Jackmin801 merged 1 commit intomainfrom
no-prompt-logprobs

Commits

Commits on Dec 26, 2025