Description
Your current environment
vllm 0.7.3
🐛 Describe the bug
Your current environment
vllm 0.7.3
🐛 Describe the bug
We encountered a bug in the LRUEvictor implementation when running VLLM (version 0.7.3) with the --preemption-mode swap flag.
The issue arises due to desynchronization between self.priority_queue and self.free_table in the remove method.
Add logging to evictor.py and prefix_caching_block.py to track block additions and removals.
The issue by observing the following sequence:
line 25: block (block_id=862 content_hash=4781171782003088483 num_hashed_tokens=768 last_accessed=-1) is added to self.free_table and self.priority_queue. (in add method)
line 26: The block is removed from self.free_table but remains in self.priority_queue. (in remove method, only self.free_table is altered)
line 28: The block is added again to self.free_table and self.priority_queue. (in add method)
line 29: The block is removed from self.free_table, and the one added in line 25 is removed from self.priority_queue. (in evict method)
line 31: block(block_id=862 content_hash=-1708738876872868168 num_hashed_tokens=192 last_accessed=-1) is added to self.free_table and self.priority_queue. note the content_hash is different。(in add method)
line 32: evict method remove the one added in line 31 from self.free_table but pop the one added in line 28 from self.priority_queue, they have different content_hash. and the error occured.
I think in evict method，we should compare all field in BlockMetaData to get the right block to return, just like blow
After making the change, we tested it and found no problems.