[Bug]: Llama4 multimodal cache missing key

[cadedaniel]

Your current environment

Python 3.11 CUDA 12.4 vLLM 0.8.4

🐛 Describe the bug

vLLM v0.8.4

ERROR 06-27 16:32:19 [core.py:387] EngineCore hit an exception: Traceback (most recent call last):
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 66, in __getitem__
ERROR 06-27 16:32:19 [core.py:387]     return self.__data[key]
ERROR 06-27 16:32:19 [core.py:387]            ~~~~~~~~~~~^^^^^
ERROR 06-27 16:32:19 [core.py:387] KeyError: 'f224b2249792ee5054e8ec7c0b68f6d22889843159c73bdea187eda555743f85'
ERROR 06-27 16:32:19 [core.py:387] 
ERROR 06-27 16:32:19 [core.py:387] During handling of the above exception, another exception occurred:
ERROR 06-27 16:32:19 [core.py:387] 
ERROR 06-27 16:32:19 [core.py:387] Traceback (most recent call last):
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 380, in run_engine_core
ERROR 06-27 16:32:19 [core.py:387]     engine_core.run_busy_loop()
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 400, in run_busy_loop
ERROR 06-27 16:32:19 [core.py:387]     self._process_input_queue()
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 425, in _process_input_queue
ERROR 06-27 16:32:19 [core.py:387]     self._handle_client_request(*req)
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 441, in _handle_client_request
ERROR 06-27 16:32:19 [core.py:387]     self.add_request(request)
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 177, in add_request
ERROR 06-27 16:32:19 [core.py:387]     request.mm_inputs = self.mm_input_cache_server.get_and_update_p1(
ERROR 06-27 16:32:19 [core.py:387]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/mm_input_cache.py", line 76, in get_and_update_p1
ERROR 06-27 16:32:19 [core.py:387]     mm_input = self.mm_cache[mm_hash]
ERROR 06-27 16:32:19 [core.py:387]                ~~~~~~~~~~~~~^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 255, in __getitem__
ERROR 06-27 16:32:19 [core.py:387]     value = cache_getitem(self, key)
ERROR 06-27 16:32:19 [core.py:387]             ^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 68, in __getitem__
ERROR 06-27 16:32:19 [core.py:387]     return self.__missing__(key)
ERROR 06-27 16:32:19 [core.py:387]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387]   File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 95, in __missing__
ERROR 06-27 16:32:19 [core.py:387]     raise KeyError(key)
ERROR 06-27 16:32:19 [core.py:387] KeyError: 'f224b2249792ee5054e8ec7c0b68f6d22889843159c73bdea187eda555743f85'
ERROR 06-27 16:32:19 [core.py:387] 
CRITICAL 06-27 16:32:19 [core_client.py:359] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

This issue should be fixed in the latest version of vLLM, can you try updating it?

[DarkLight1337]

In particular it should be fixed by #16593