Open
Description
Proposal to improve performance
No response
Report of performance regression
No response
Misc discussion on performance
I wonder how dynamic batching will perform after enabling the disaggregated prefill? Could anyone inform me about where is the relevant scheduling code.
Your current environment (if you think it is necessary)
The output of `python collect_env.py`
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.