[Core] Use CpuGpuBuffer for block table tensors
#24795
Conversation
Signed-off-by: Nick Hill <nhill@redhat.com>
njhill
requested review from
WoosukKwon,
alexm-redhat,
comaniac,
robertgshaw2-redhat and
ywang96
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the BlockTable class to use the CpuGpuBuffer helper for managing tensors that exist on both CPU and GPU, specifically for block_table and slot_mapping. This change simplifies the code by centralizing the creation and management of these paired buffers, removing redundant manual handling of CPU, GPU, and NumPy tensor versions. The changes are well-contained within vllm/v1/worker/block_table.py, and the necessary adjustments in vllm/v1/worker/gpu_model_runner.py to adapt to the updated BlockTable API have been correctly applied. The refactoring improves code clarity and maintainability without introducing any apparent issues.
There was a problem hiding this comment.
cc @WoosukKwon please confirm if this is ok to avoid conflicting with your refactoring efforts
Pull Request is not mergeable
Signed-off-by: Nick Hill <nhill@redhat.com>
|
The failing test is also failing on main :( example: https://buildkite.com/vllm/ci/builds/30944#01995329-1ec7-4d17-abdf-0283fa8115f5 |
vllm-project/vllm#24795 and vllm-project/vllm#24615 and vllm-project/vllm#24078 --------- Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
vllm-project/vllm#24795 and vllm-project/vllm#24615 and vllm-project/vllm#24078 --------- Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai> Signed-off-by: slokesha <slokeshappa@habana.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
4 participants
In the gpu model runner input batch.