Unable to pass images to Aya Vision processor

[DarkLight1337]

System Info

transformers 4.52

Who can help?

@zucchini-nlp @hmellor

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

In the new version of transformers, _check_special_mm_tokens is being called inside AyaVisionProcessor. However, _check_special_mm_tokens assumes that the image placeholder <image> can be represented as a single token. This is not the case for Aya Vision 8B which encodes <image> into [35, 6504, 37]. As a result, the validation always fails whenever an image is passed.

I discovered this issue when attempting to update transformers version in vLLM: vllm-project/vllm#18678

Error log: https://buildkite.com/vllm/fastcheck/builds/25098/steps?sid=019706c6-1a33-4922-9358-d72dfc525fe2 https://buildkite.com/vllm/fastcheck/builds/25098/steps?sid=019706c6-1a35-46ac-aa2b-8d6d811109fd

Expected behavior

_check_special_mm_tokens should handle the case where the modality text takes up multiple tokens.

[zucchini-nlp]

Huh weird that the tests didn't fail for this. Having a look 👀

[zucchini-nlp]

The failing tests I see are not related to AyaVision. I see a few for Idefics3 caused by mentioned PR, as I changed the self.image_token to be a string. Also rechecked that the tests on transformers are green. Can you point me to the specific test failure?

Btw, the image_token is indeed not a single token and actually _check_special_mm_tokens isn't checking anything because it is replaced by an img_patch_token. Ideally I aim to use the same token as placeholder, yet in case of Aya it will be quite breaking. Thus the _check_special_mm_tokens is more like a no-op and I don't expect it to affect anything

[DarkLight1337]

Sorry the link was incorrect. It should be this one: https://buildkite.com/vllm/fastcheck/builds/25098/steps?sid=019706c6-1a35-46ac-aa2b-8d6d811109fd

[zucchini-nlp]

@DarkLight1337 sorry, I can't see the aya vision failure in MM test suite, is it possible to maybe copy the traceback?

[DarkLight1337]

Raw logs:

[2025-05-25T12:05:35Z] =================================== FAILURES ===================================
[2025-05-25T12:05:35Z] �[31m�[1m_______________ test_single_image_models[aya_vision-test_case12] _______________�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_single_image_models_aya_v0')
[2025-05-25T12:05:35Z] model_type = 'aya_vision'
[2025-05-25T12:05:35Z] test_case = ExpandableVLMTestArgs(model='CohereForAI/aya-vision-8b', max_tokens=128, num_logprobs=5, dtype='auto', distributed_exe..., size_wrapper=ImageSizeWrapper(type=<SizeType.SIZE_FACTOR: 1>, data=()), num_video_frames=None, custom_test_opts=None)
[2025-05-25T12:05:35Z] hf_runner = <class 'tests.conftest.HfRunner'>
[2025-05-25T12:05:35Z] vllm_runner = <class 'tests.conftest.VllmRunner'>
[2025-05-25T12:05:35Z] image_assets = [ImageAsset(name='stop_sign'), ImageAsset(name='cherry_blossom')]
[2025-05-25T12:05:35Z] monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff6ce624920>
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[37m@pytest�[39;49;00m.mark.parametrize(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[33m"�[39;49;00m�[33mmodel_type,test_case�[39;49;00m�[33m"�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         get_parametrized_options(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             VLM_TEST_SETTINGS,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_type=VLMTestType.IMAGE,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             create_new_process_for_each_test=�[94mFalse�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         ))�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[94mdef�[39;49;00m �[92mtest_single_image_models�[39;49;00m(tmp_path: PosixPath, model_type: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  test_case: ExpandableVLMTestArgs,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  hf_runner: �[96mtype�[39;49;00m[HfRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  vllm_runner: �[96mtype�[39;49;00m[VllmRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  image_assets: ImageTestAssets, monkeypatch):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m model_type �[95min�[39;49;00m REQUIRES_V0_MODELS:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             monkeypatch.setenv(�[33m"�[39;49;00m�[33mVLLM_USE_V1�[39;49;00m�[33m"�[39;49;00m, �[33m"�[39;49;00m�[33m0�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         model_test_info = VLM_TEST_SETTINGS[model_type]�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >       runners.run_single_image_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             tmp_path=tmp_path,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             model_test_info=model_test_info,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_case=test_case,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             hf_runner=hf_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             vllm_runner=vllm_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             image_assets=image_assets,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         )�[90m�[39;49;00m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/test_common.py�[0m:747: 
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/runners.py�[0m:23: in run_single_image_test
[2025-05-25T12:05:35Z]     �[0mcore.run_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/core.py�[0m:77: in run_test
[2025-05-25T12:05:35Z]     �[0m�[94mwith�[39;49;00m vllm_runner(model,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mconftest.py�[0m:771: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.model = LLM(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/utils.py�[0m:1184: in inner
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m fn(*args, **kwargs)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/llm.py�[0m:250: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.llm_engine = LLMEngine.from_engine_args(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/engine/llm_engine.py�[0m:501: in from_engine_args
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m engine_cls.from_vllm_config(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:115: in from_vllm_config
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m �[96mcls�[39;49;00m(vllm_config=vllm_config,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:92: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.engine_core = EngineCoreClient.make_client(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:75: in make_client
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m SyncMPClient(vllm_config, executor_class, log_stats)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:580: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96msuper�[39;49;00m().�[92m__init__�[39;49;00m(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:418: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m._wait_for_engine_startup(output_address, parallel_config)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] self = <vllm.v1.engine.core_client.SyncMPClient object at 0x7ff6cc8a38f0>
[2025-05-25T12:05:35Z] output_address = 'ipc:///tmp/4bad6597-8bb3-4a82-87be-fd913ef94652'
[2025-05-25T12:05:35Z] parallel_config = ParallelConfig(pipeline_parallel_size=1, tensor_parallel_size=1, data_parallel_size=1, data_parallel_size_local=1, dat...i', worker_cls='vllm.v1.worker.gpu_worker.Worker', sd_worker_cls='auto', worker_extension_cls='', world_size=1, rank=0)
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[94mdef�[39;49;00m �[92m_wait_for_engine_startup�[39;49;00m(�[96mself�[39;49;00m, output_address: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  parallel_config: ParallelConfig):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Get a sync handle to the socket which can be sync or async.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         sync_input_socket = zmq.Socket.shadow(�[96mself�[39;49;00m.input_socket)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Wait for engine core process(es) to send ready messages.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         local_count = parallel_config.data_parallel_size_local�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         remote_count = �[96mlen�[39;49;00m(�[96mself�[39;49;00m.core_engines) - local_count�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# [local, remote] counts�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         conn_pending, start_pending = [local_count, remote_count], [�[94m0�[39;49;00m, �[94m0�[39;49;00m]�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller = zmq.Poller()�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller.register(sync_input_socket, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         proc_manager = �[96mself�[39;49;00m.resources.local_engine_manager�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m proc_manager �[95mis�[39;49;00m �[95mnot�[39;49;00m �[94mNone�[39;49;00m:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mfor�[39;49;00m sentinel �[95min�[39;49;00m proc_manager.sentinels():�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 poller.register(sentinel, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mwhile�[39;49;00m �[96many�[39;49;00m(conn_pending) �[95mor�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             events = poller.poll(STARTUP_POLL_PERIOD_MS)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[95mnot�[39;49;00m events:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(conn_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto connect.�[39;49;00m�[33m"�[39;49;00m, *conn_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto start.�[39;49;00m�[33m"�[39;49;00m, *start_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mcontinue�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[96mlen�[39;49;00m(events) > �[94m1�[39;49;00m �[95mor�[39;49;00m events[�[94m0�[39;49;00m][�[94m0�[39;49;00m] != sync_input_socket:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[90m# One of the local core processes exited.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 finished = proc_manager.finished_procs(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 ) �[94mif�[39;49;00m proc_manager �[94melse�[39;49;00m {}�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >               �[94mraise�[39;49;00m �[96mRuntimeError�[39;49;00m(�[33m"�[39;49;00m�[33mEngine core initialization failed. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33m"�[39;49;00m�[33mSee root cause above. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33mf�[39;49;00m�[33m"�[39;49;00m�[33mFailed core proc(s): �[39;49;00m�[33m{�[39;49;00mfinished�[33m}�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mE               RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:484: RuntimeError
[2025-05-25T12:05:35Z] �[31m�[1m_______________ test_single_image_models[aya_vision-test_case13] _______________�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_single_image_models_aya_v1')
[2025-05-25T12:05:35Z] model_type = 'aya_vision'
[2025-05-25T12:05:35Z] test_case = ExpandableVLMTestArgs(model='CohereForAI/aya-vision-8b', max_tokens=128, num_logprobs=5, dtype='auto', distributed_exe...ze_wrapper=ImageSizeWrapper(type=<SizeType.SIZE_FACTOR: 1>, data=(1.0,)), num_video_frames=None, custom_test_opts=None)
[2025-05-25T12:05:35Z] hf_runner = <class 'tests.conftest.HfRunner'>
[2025-05-25T12:05:35Z] vllm_runner = <class 'tests.conftest.VllmRunner'>
[2025-05-25T12:05:35Z] image_assets = [ImageAsset(name='stop_sign'), ImageAsset(name='cherry_blossom')]
[2025-05-25T12:05:35Z] monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff6ce624260>
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[37m@pytest�[39;49;00m.mark.parametrize(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[33m"�[39;49;00m�[33mmodel_type,test_case�[39;49;00m�[33m"�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         get_parametrized_options(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             VLM_TEST_SETTINGS,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_type=VLMTestType.IMAGE,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             create_new_process_for_each_test=�[94mFalse�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         ))�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[94mdef�[39;49;00m �[92mtest_single_image_models�[39;49;00m(tmp_path: PosixPath, model_type: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  test_case: ExpandableVLMTestArgs,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  hf_runner: �[96mtype�[39;49;00m[HfRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  vllm_runner: �[96mtype�[39;49;00m[VllmRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  image_assets: ImageTestAssets, monkeypatch):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m model_type �[95min�[39;49;00m REQUIRES_V0_MODELS:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             monkeypatch.setenv(�[33m"�[39;49;00m�[33mVLLM_USE_V1�[39;49;00m�[33m"�[39;49;00m, �[33m"�[39;49;00m�[33m0�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         model_test_info = VLM_TEST_SETTINGS[model_type]�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >       runners.run_single_image_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             tmp_path=tmp_path,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             model_test_info=model_test_info,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_case=test_case,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             hf_runner=hf_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             vllm_runner=vllm_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             image_assets=image_assets,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         )�[90m�[39;49;00m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/test_common.py�[0m:747: 
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/runners.py�[0m:23: in run_single_image_test
[2025-05-25T12:05:35Z]     �[0mcore.run_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/core.py�[0m:77: in run_test
[2025-05-25T12:05:35Z]     �[0m�[94mwith�[39;49;00m vllm_runner(model,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mconftest.py�[0m:771: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.model = LLM(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/utils.py�[0m:1184: in inner
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m fn(*args, **kwargs)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/llm.py�[0m:250: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.llm_engine = LLMEngine.from_engine_args(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/engine/llm_engine.py�[0m:501: in from_engine_args
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m engine_cls.from_vllm_config(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:115: in from_vllm_config
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m �[96mcls�[39;49;00m(vllm_config=vllm_config,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:92: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.engine_core = EngineCoreClient.make_client(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:75: in make_client
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m SyncMPClient(vllm_config, executor_class, log_stats)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:580: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96msuper�[39;49;00m().�[92m__init__�[39;49;00m(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:418: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m._wait_for_engine_startup(output_address, parallel_config)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] self = <vllm.v1.engine.core_client.SyncMPClient object at 0x7ff6cc8ce750>
[2025-05-25T12:05:35Z] output_address = 'ipc:///tmp/ed684d10-72cc-40dc-89d7-8c423243d76e'
[2025-05-25T12:05:35Z] parallel_config = ParallelConfig(pipeline_parallel_size=1, tensor_parallel_size=1, data_parallel_size=1, data_parallel_size_local=1, dat...i', worker_cls='vllm.v1.worker.gpu_worker.Worker', sd_worker_cls='auto', worker_extension_cls='', world_size=1, rank=0)
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[94mdef�[39;49;00m �[92m_wait_for_engine_startup�[39;49;00m(�[96mself�[39;49;00m, output_address: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  parallel_config: ParallelConfig):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Get a sync handle to the socket which can be sync or async.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         sync_input_socket = zmq.Socket.shadow(�[96mself�[39;49;00m.input_socket)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Wait for engine core process(es) to send ready messages.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         local_count = parallel_config.data_parallel_size_local�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         remote_count = �[96mlen�[39;49;00m(�[96mself�[39;49;00m.core_engines) - local_count�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# [local, remote] counts�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         conn_pending, start_pending = [local_count, remote_count], [�[94m0�[39;49;00m, �[94m0�[39;49;00m]�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller = zmq.Poller()�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller.register(sync_input_socket, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         proc_manager = �[96mself�[39;49;00m.resources.local_engine_manager�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m proc_manager �[95mis�[39;49;00m �[95mnot�[39;49;00m �[94mNone�[39;49;00m:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mfor�[39;49;00m sentinel �[95min�[39;49;00m proc_manager.sentinels():�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 poller.register(sentinel, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mwhile�[39;49;00m �[96many�[39;49;00m(conn_pending) �[95mor�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             events = poller.poll(STARTUP_POLL_PERIOD_MS)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[95mnot�[39;49;00m events:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(conn_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto connect.�[39;49;00m�[33m"�[39;49;00m, *conn_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto start.�[39;49;00m�[33m"�[39;49;00m, *start_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mcontinue�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[96mlen�[39;49;00m(events) > �[94m1�[39;49;00m �[95mor�[39;49;00m events[�[94m0�[39;49;00m][�[94m0�[39;49;00m] != sync_input_socket:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[90m# One of the local core processes exited.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 finished = proc_manager.finished_procs(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 ) �[94mif�[39;49;00m proc_manager �[94melse�[39;49;00m {}�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >               �[94mraise�[39;49;00m �[96mRuntimeError�[39;49;00m(�[33m"�[39;49;00m�[33mEngine core initialization failed. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33m"�[39;49;00m�[33mSee root cause above. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33mf�[39;49;00m�[33m"�[39;49;00m�[33mFailed core proc(s): �[39;49;00m�[33m{�[39;49;00mfinished�[33m}�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mE               RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:484: RuntimeError
[2025-05-25T12:05:35Z] �[31m�[1m_______________ test_single_image_models[aya_vision-test_case14] _______________�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_single_image_models_aya_v2')
[2025-05-25T12:05:35Z] model_type = 'aya_vision'
[2025-05-25T12:05:35Z] test_case = ExpandableVLMTestArgs(model='CohereForAI/aya-vision-8b', max_tokens=128, num_logprobs=5, dtype='auto', distributed_exe...r=ImageSizeWrapper(type=<SizeType.SIZE_FACTOR: 1>, data=(1.0, 1.0, 1.0)), num_video_frames=None, custom_test_opts=None)
[2025-05-25T12:05:35Z] hf_runner = <class 'tests.conftest.HfRunner'>
[2025-05-25T12:05:35Z] vllm_runner = <class 'tests.conftest.VllmRunner'>
[2025-05-25T12:05:35Z] image_assets = [ImageAsset(name='stop_sign'), ImageAsset(name='cherry_blossom')]
[2025-05-25T12:05:35Z] monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff6cb63fb90>
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[37m@pytest�[39;49;00m.mark.parametrize(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[33m"�[39;49;00m�[33mmodel_type,test_case�[39;49;00m�[33m"�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         get_parametrized_options(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             VLM_TEST_SETTINGS,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_type=VLMTestType.IMAGE,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             create_new_process_for_each_test=�[94mFalse�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         ))�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[94mdef�[39;49;00m �[92mtest_single_image_models�[39;49;00m(tmp_path: PosixPath, model_type: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  test_case: ExpandableVLMTestArgs,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  hf_runner: �[96mtype�[39;49;00m[HfRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  vllm_runner: �[96mtype�[39;49;00m[VllmRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  image_assets: ImageTestAssets, monkeypatch):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m model_type �[95min�[39;49;00m REQUIRES_V0_MODELS:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             monkeypatch.setenv(�[33m"�[39;49;00m�[33mVLLM_USE_V1�[39;49;00m�[33m"�[39;49;00m, �[33m"�[39;49;00m�[33m0�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         model_test_info = VLM_TEST_SETTINGS[model_type]�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >       runners.run_single_image_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             tmp_path=tmp_path,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             model_test_info=model_test_info,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_case=test_case,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             hf_runner=hf_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             vllm_runner=vllm_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             image_assets=image_assets,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         )�[90m�[39;49;00m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/test_common.py�[0m:747: 
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/runners.py�[0m:23: in run_single_image_test
[2025-05-25T12:05:35Z]     �[0mcore.run_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/core.py�[0m:77: in run_test
[2025-05-25T12:05:35Z]     �[0m�[94mwith�[39;49;00m vllm_runner(model,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mconftest.py�[0m:771: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.model = LLM(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/utils.py�[0m:1184: in inner
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m fn(*args, **kwargs)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/llm.py�[0m:250: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.llm_engine = LLMEngine.from_engine_args(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/engine/llm_engine.py�[0m:501: in from_engine_args
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m engine_cls.from_vllm_config(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:115: in from_vllm_config
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m �[96mcls�[39;49;00m(vllm_config=vllm_config,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:92: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.engine_core = EngineCoreClient.make_client(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:75: in make_client
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m SyncMPClient(vllm_config, executor_class, log_stats)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:580: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96msuper�[39;49;00m().�[92m__init__�[39;49;00m(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:418: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m._wait_for_engine_startup(output_address, parallel_config)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] self = <vllm.v1.engine.core_client.SyncMPClient object at 0x7ff6cd179220>
[2025-05-25T12:05:35Z] output_address = 'ipc:///tmp/1f9ab11e-be64-4854-9215-9008e04e16a8'
[2025-05-25T12:05:35Z] parallel_config = ParallelConfig(pipeline_parallel_size=1, tensor_parallel_size=1, data_parallel_size=1, data_parallel_size_local=1, dat...i', worker_cls='vllm.v1.worker.gpu_worker.Worker', sd_worker_cls='auto', worker_extension_cls='', world_size=1, rank=0)
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[94mdef�[39;49;00m �[92m_wait_for_engine_startup�[39;49;00m(�[96mself�[39;49;00m, output_address: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  parallel_config: ParallelConfig):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Get a sync handle to the socket which can be sync or async.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         sync_input_socket = zmq.Socket.shadow(�[96mself�[39;49;00m.input_socket)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Wait for engine core process(es) to send ready messages.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         local_count = parallel_config.data_parallel_size_local�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         remote_count = �[96mlen�[39;49;00m(�[96mself�[39;49;00m.core_engines) - local_count�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# [local, remote] counts�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         conn_pending, start_pending = [local_count, remote_count], [�[94m0�[39;49;00m, �[94m0�[39;49;00m]�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller = zmq.Poller()�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller.register(sync_input_socket, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         proc_manager = �[96mself�[39;49;00m.resources.local_engine_manager�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m proc_manager �[95mis�[39;49;00m �[95mnot�[39;49;00m �[94mNone�[39;49;00m:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mfor�[39;49;00m sentinel �[95min�[39;49;00m proc_manager.sentinels():�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 poller.register(sentinel, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mwhile�[39;49;00m �[96many�[39;49;00m(conn_pending) �[95mor�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             events = poller.poll(STARTUP_POLL_PERIOD_MS)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[95mnot�[39;49;00m events:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(conn_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto connect.�[39;49;00m�[33m"�[39;49;00m, *conn_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto start.�[39;49;00m�[33m"�[39;49;00m, *start_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mcontinue�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[96mlen�[39;49;00m(events) > �[94m1�[39;49;00m �[95mor�[39;49;00m events[�[94m0�[39;49;00m][�[94m0�[39;49;00m] != sync_input_socket:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[90m# One of the local core processes exited.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 finished = proc_manager.finished_procs(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 ) �[94mif�[39;49;00m proc_manager �[94melse�[39;49;00m {}�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >               �[94mraise�[39;49;00m �[96mRuntimeError�[39;49;00m(�[33m"�[39;49;00m�[33mEngine core initialization failed. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33m"�[39;49;00m�[33mSee root cause above. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33mf�[39;49;00m�[33m"�[39;49;00m�[33mFailed core proc(s): �[39;49;00m�[33m{�[39;49;00mfinished�[33m}�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mE               RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:484: RuntimeError
[2025-05-25T12:05:35Z] �[31m�[1m_______________ test_single_image_models[aya_vision-test_case15] _______________�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_single_image_models_aya_v3')
[2025-05-25T12:05:35Z] model_type = 'aya_vision'
[2025-05-25T12:05:35Z] test_case = ExpandableVLMTestArgs(model='CohereForAI/aya-vision-8b', max_tokens=128, num_logprobs=5, dtype='auto', distributed_exe...=ImageSizeWrapper(type=<SizeType.SIZE_FACTOR: 1>, data=(0.25, 0.5, 1.0)), num_video_frames=None, custom_test_opts=None)
[2025-05-25T12:05:35Z] hf_runner = <class 'tests.conftest.HfRunner'>
[2025-05-25T12:05:35Z] vllm_runner = <class 'tests.conftest.VllmRunner'>
[2025-05-25T12:05:35Z] image_assets = [ImageAsset(name='stop_sign'), ImageAsset(name='cherry_blossom')]
[2025-05-25T12:05:35Z] monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7ff6cc794830>
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[37m@pytest�[39;49;00m.mark.parametrize(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[33m"�[39;49;00m�[33mmodel_type,test_case�[39;49;00m�[33m"�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         get_parametrized_options(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             VLM_TEST_SETTINGS,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_type=VLMTestType.IMAGE,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             create_new_process_for_each_test=�[94mFalse�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         ))�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[94mdef�[39;49;00m �[92mtest_single_image_models�[39;49;00m(tmp_path: PosixPath, model_type: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  test_case: ExpandableVLMTestArgs,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  hf_runner: �[96mtype�[39;49;00m[HfRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  vllm_runner: �[96mtype�[39;49;00m[VllmRunner],�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  image_assets: ImageTestAssets, monkeypatch):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m model_type �[95min�[39;49;00m REQUIRES_V0_MODELS:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             monkeypatch.setenv(�[33m"�[39;49;00m�[33mVLLM_USE_V1�[39;49;00m�[33m"�[39;49;00m, �[33m"�[39;49;00m�[33m0�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         model_test_info = VLM_TEST_SETTINGS[model_type]�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >       runners.run_single_image_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             tmp_path=tmp_path,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             model_test_info=model_test_info,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             test_case=test_case,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             hf_runner=hf_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             vllm_runner=vllm_runner,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             image_assets=image_assets,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         )�[90m�[39;49;00m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/test_common.py�[0m:747: 
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/runners.py�[0m:23: in run_single_image_test
[2025-05-25T12:05:35Z]     �[0mcore.run_test(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mmodels/multimodal/generation/vlm_utils/core.py�[0m:77: in run_test
[2025-05-25T12:05:35Z]     �[0m�[94mwith�[39;49;00m vllm_runner(model,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mconftest.py�[0m:771: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.model = LLM(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/utils.py�[0m:1184: in inner
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m fn(*args, **kwargs)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/llm.py�[0m:250: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.llm_engine = LLMEngine.from_engine_args(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/engine/llm_engine.py�[0m:501: in from_engine_args
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m engine_cls.from_vllm_config(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:115: in from_vllm_config
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m �[96mcls�[39;49;00m(vllm_config=vllm_config,�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/llm_engine.py�[0m:92: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m.engine_core = EngineCoreClient.make_client(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:75: in make_client
[2025-05-25T12:05:35Z]     �[0m�[94mreturn�[39;49;00m SyncMPClient(vllm_config, executor_class, log_stats)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:580: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96msuper�[39;49;00m().�[92m__init__�[39;49;00m(�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:418: in __init__
[2025-05-25T12:05:35Z]     �[0m�[96mself�[39;49;00m._wait_for_engine_startup(output_address, parallel_config)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] self = <vllm.v1.engine.core_client.SyncMPClient object at 0x7ff6cb79dd30>
[2025-05-25T12:05:35Z] output_address = 'ipc:///tmp/1910d71d-19db-4360-8d8c-5ea27cde3e4e'
[2025-05-25T12:05:35Z] parallel_config = ParallelConfig(pipeline_parallel_size=1, tensor_parallel_size=1, data_parallel_size=1, data_parallel_size_local=1, dat...i', worker_cls='vllm.v1.worker.gpu_worker.Worker', sd_worker_cls='auto', worker_extension_cls='', world_size=1, rank=0)
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z]     �[0m�[94mdef�[39;49;00m �[92m_wait_for_engine_startup�[39;49;00m(�[96mself�[39;49;00m, output_address: �[96mstr�[39;49;00m,�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                  parallel_config: ParallelConfig):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Get a sync handle to the socket which can be sync or async.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         sync_input_socket = zmq.Socket.shadow(�[96mself�[39;49;00m.input_socket)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# Wait for engine core process(es) to send ready messages.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         local_count = parallel_config.data_parallel_size_local�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         remote_count = �[96mlen�[39;49;00m(�[96mself�[39;49;00m.core_engines) - local_count�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[90m# [local, remote] counts�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         conn_pending, start_pending = [local_count, remote_count], [�[94m0�[39;49;00m, �[94m0�[39;49;00m]�[90m�[39;49;00m
[2025-05-25T12:05:35Z]     �[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller = zmq.Poller()�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         poller.register(sync_input_socket, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         proc_manager = �[96mself�[39;49;00m.resources.local_engine_manager�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mif�[39;49;00m proc_manager �[95mis�[39;49;00m �[95mnot�[39;49;00m �[94mNone�[39;49;00m:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mfor�[39;49;00m sentinel �[95min�[39;49;00m proc_manager.sentinels():�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 poller.register(sentinel, zmq.POLLIN)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]         �[94mwhile�[39;49;00m �[96many�[39;49;00m(conn_pending) �[95mor�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             events = poller.poll(STARTUP_POLL_PERIOD_MS)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[95mnot�[39;49;00m events:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(conn_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto connect.�[39;49;00m�[33m"�[39;49;00m, *conn_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mif�[39;49;00m �[96many�[39;49;00m(start_pending):�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                     logger.debug(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mWaiting for �[39;49;00m�[33m%d�[39;49;00m�[33m local, �[39;49;00m�[33m%d�[39;49;00m�[33m remote core engine proc(s) �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                         �[33m"�[39;49;00m�[33mto start.�[39;49;00m�[33m"�[39;49;00m, *start_pending)�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[94mcontinue�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]             �[94mif�[39;49;00m �[96mlen�[39;49;00m(events) > �[94m1�[39;49;00m �[95mor�[39;49;00m events[�[94m0�[39;49;00m][�[94m0�[39;49;00m] != sync_input_socket:�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 �[90m# One of the local core processes exited.�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 finished = proc_manager.finished_procs(�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                 ) �[94mif�[39;49;00m proc_manager �[94melse�[39;49;00m {}�[90m�[39;49;00m
[2025-05-25T12:05:35Z] >               �[94mraise�[39;49;00m �[96mRuntimeError�[39;49;00m(�[33m"�[39;49;00m�[33mEngine core initialization failed. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33m"�[39;49;00m�[33mSee root cause above. �[39;49;00m�[33m"�[39;49;00m�[90m�[39;49;00m
[2025-05-25T12:05:35Z]                                    �[33mf�[39;49;00m�[33m"�[39;49;00m�[33mFailed core proc(s): �[39;49;00m�[33m{�[39;49;00mfinished�[33m}�[39;49;00m�[33m"�[39;49;00m)�[90m�[39;49;00m
[2025-05-25T12:05:35Z] �[1m�[31mE               RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}�[0m
[2025-05-25T12:05:35Z] 
[2025-05-25T12:05:35Z] �[1m�[31m/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py�[0m:484: RuntimeError
[2025-05-25T12:05:35Z] �[36m�[1m=========================== short test summary info ============================�[0m
[2025-05-25T12:05:35Z] �[31mFAILED�[0m models/multimodal/generation/test_common.py::�[1mtest_single_image_models[aya_vision-test_case12]�[0m - RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
[2025-05-25T12:05:35Z] �[31mFAILED�[0m models/multimodal/generation/test_common.py::�[1mtest_single_image_models[aya_vision-test_case13]�[0m - RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
[2025-05-25T12:05:35Z] �[31mFAILED�[0m models/multimodal/generation/test_common.py::�[1mtest_single_image_models[aya_vision-test_case14]�[0m - RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
[2025-05-25T12:05:35Z] �[31mFAILED�[0m models/multimodal/generation/test_common.py::�[1mtest_single_image_models[aya_vision-test_case15]�[0m - RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

[DarkLight1337]

The stack trace for one of the failures:

[2025-05-25T10:27:27Z] models/multimodal/generation/test_common.py::test_single_image_models[aya_vision-test_case12] INFO 05-25 03:27:27 [config.py:788] This model supports multiple tasks: {'generate', 'classify', 'reward', 'embed', 'score'}. Defaulting to 'generate'.
[2025-05-25T10:27:27Z] DEBUG 05-25 03:27:27 [arg_utils.py:1549] Setting max_num_batched_tokens to 8192 for LLM_CLASS usage context.
[2025-05-25T10:27:27Z] INFO 05-25 03:27:27 [config.py:2112] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2025-05-25T10:27:27Z] WARNING 05-25 03:27:27 [cuda.py:87] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:28] No plugins for group vllm.platform_plugins found.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:34] Checking if TPU platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:44] TPU platform is not available because: No module named 'libtpu'
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:51] Checking if CUDA platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:71] Confirmed CUDA platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:99] Checking if ROCm platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:113] ROCm platform is not available because: No module named 'amdsmi'
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:120] Checking if HPU platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:127] HPU platform is not available because habana_frameworks is not found.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:137] Checking if XPU platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:147] XPU platform is not available because: No module named 'intel_extension_for_pytorch'
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:154] Checking if CPU platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:176] Checking if Neuron platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:51] Checking if CUDA platform is available.
[2025-05-25T10:27:32Z] DEBUG 05-25 03:27:32 [__init__.py:71] Confirmed CUDA platform is available.
[2025-05-25T10:27:32Z] INFO 05-25 03:27:32 [__init__.py:243] Automatically detected platform cuda.
[2025-05-25T10:27:35Z] INFO 05-25 03:27:35 [core.py:438] Waiting for init message from front-end.
[2025-05-25T10:27:35Z] DEBUG 05-25 03:27:35 [core_client.py:540] HELLO from local core engine process 0.
[2025-05-25T10:27:35Z] DEBUG 05-25 03:27:35 [core.py:445] Received init message: {'output_socket_address': 'ipc:///tmp/4bad6597-8bb3-4a82-87be-fd913ef94652', 'parallel_config': {'data_parallel_master_ip': '127.0.0.1', 'data_parallel_master_port': 0, 'data_parallel_size': 1}}
[2025-05-25T10:27:35Z] INFO 05-25 03:27:35 [__init__.py:31] Available plugins for group vllm.general_plugins:
[2025-05-25T10:27:35Z] INFO 05-25 03:27:35 [__init__.py:33] - lora_filesystem_resolver -> vllm.plugins.lora_resolvers.filesystem_resolver:register_filesystem_resolver
[2025-05-25T10:27:35Z] INFO 05-25 03:27:35 [__init__.py:36] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
[2025-05-25T10:27:35Z] INFO 05-25 03:27:35 [core.py:65] Initializing a V1 LLM engine (v0.8.5.dev914+g0cabf750f) with config: model='CohereForAI/aya-vision-8b', speculative_config=None, tokenizer='CohereForAI/aya-vision-8b', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=CohereForAI/aya-vision-8b, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=False, pooler_config=None, compilation_config={"compile_sizes": [], "inductor_compile_config": {"enable_auto_functionalized_v2": false}, "cudagraph_capture_sizes": [], "max_capture_size": 0}
[2025-05-25T10:27:35Z] DEBUG 05-25 03:27:35 [decorators.py:109] Inferred dynamic dimensions for forward method of <class 'vllm.model_executor.models.llama.LlamaModel'>: ['input_ids', 'positions', 'intermediate_tensors', 'inputs_embeds']
[2025-05-25T10:27:35Z] DEBUG 05-25 03:27:35 [decorators.py:109] Inferred dynamic dimensions for forward method of <class 'vllm.model_executor.models.llama_eagle3.LlamaModel'>: ['input_ids', 'positions', 'hidden_states']
[2025-05-25T10:27:35Z] WARNING 05-25 03:27:35 [utils.py:2671] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes,initialize_cache not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x7fbda612bad0>
[2025-05-25T10:27:35Z] DEBUG 05-25 03:27:35 [config.py:4521] enabled custom ops: Counter()
[2025-05-25T10:27:35Z] DEBUG 05-25 03:27:35 [config.py:4523] disabled custom ops: Counter()
[2025-05-25T10:27:36Z] DEBUG 05-25 03:27:36 [parallel_state.py:917] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://172.16.0.2:60653 backend=nccl
[2025-05-25T10:27:36Z] INFO 05-25 03:27:36 [parallel_state.py:1064] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
[2025-05-25T10:27:37Z] Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] EngineCore failed to start.

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] Traceback (most recent call last):

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 157, in call_hf_processor

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     return hf_processor(**data, **merged_kwargs, return_tensors="pt")

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/transformers/models/aya_vision/processing_aya_vision.py", line 230, in __call__

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self._check_special_mm_tokens(text, text_inputs, modalities=["image"])

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/transformers/processing_utils.py", line 1665, in _check_special_mm_tokens

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     raise ValueError(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] ValueError: Mismatch in `image` token count between text and `input_ids`. Got ids=[0] and text=[1]. Likely due to `truncation='max_length'`. Please disable truncation or increase `max_length`.

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] 

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] The above exception was the direct cause of the following exception:

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] 

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] Traceback (most recent call last):

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 491, in run_engine_core

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     engine_core = EngineCoreProc(*args, **kwargs)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 390, in __init__

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     super().__init__(vllm_config, executor_class, log_stats,

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 71, in __init__

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self.model_executor = executor_class(vllm_config)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/executor_base.py", line 52, in __init__

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self._init_executor()

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 46, in _init_executor

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self.collective_rpc("init_device")

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     answer = run_method(self.driver_worker, method, args, kwargs)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/utils.py", line 2605, in run_method

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     return func(*args, **kwargs)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]            ^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 604, in init_device

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self.worker.init_device()  # type: ignore

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     ^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 144, in init_device

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self.model_runner: GPUModelRunner = GPUModelRunner(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                                         ^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 126, in __init__

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     encoder_compute_budget, encoder_cache_size = compute_encoder_budget(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                                                  ^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 94, in compute_encoder_budget

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     ) = _compute_encoder_budget_multimodal(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 124, in _compute_encoder_budget_multimodal

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     .get_max_tokens_per_item_by_nonzero_modality(model_config)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 147, in get_max_tokens_per_item_by_nonzero_modality

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     self.get_max_tokens_per_item_by_modality(model_config).items()

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 121, in get_max_tokens_per_item_by_modality

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     return profiler.get_mm_max_tokens(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 266, in get_mm_max_tokens

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 179, in _get_dummy_mm_inputs

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     return self.processor.apply(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]            ^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1763, in apply

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     ) = self._cached_apply_hf_processor(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1529, in _cached_apply_hf_processor

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     ) = self._apply_hf_processor_main(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1372, in _apply_hf_processor_main

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     prompt_ids = self._apply_hf_processor_text_only(prompt)

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1300, in _apply_hf_processor_text_only

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     prompt_ids, _, _ = self._apply_hf_processor_text_mm(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1270, in _apply_hf_processor_text_mm

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     processed_data = self._call_hf_processor(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                      ^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/aya_vision.py", line 181, in _call_hf_processor

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     processed_outputs = super()._call_hf_processor(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1233, in _call_hf_processor

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     return self.info.ctx.call_hf_processor(

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]   File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 162, in call_hf_processor

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500]     raise ValueError(msg) from exc

[2025-05-25T10:27:41Z] ERROR 05-25 03:27:41 [core.py:500] ValueError: Failed to apply AyaVisionProcessor on data={'text': '<image>'} with kwargs={'crop_to_patches': True}
[2025-05-25T10:27:41Z] Process EngineCore_0:
[2025-05-25T10:27:41Z] Traceback (most recent call last):
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 157, in call_hf_processor
[2025-05-25T10:27:41Z]     return hf_processor(**data, **merged_kwargs, return_tensors="pt")
[2025-05-25T10:27:41Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/transformers/models/aya_vision/processing_aya_vision.py", line 230, in __call__
[2025-05-25T10:27:41Z]     self._check_special_mm_tokens(text, text_inputs, modalities=["image"])
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/transformers/processing_utils.py", line 1665, in _check_special_mm_tokens
[2025-05-25T10:27:41Z]     raise ValueError(
[2025-05-25T10:27:41Z] ValueError: Mismatch in `image` token count between text and `input_ids`. Got ids=[0] and text=[1]. Likely due to `truncation='max_length'`. Please disable truncation or increase `max_length`.
[2025-05-25T10:27:41Z] 
[2025-05-25T10:27:41Z] The above exception was the direct cause of the following exception:
[2025-05-25T10:27:41Z] 
[2025-05-25T10:27:41Z] Traceback (most recent call last):
[2025-05-25T10:27:41Z]   File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
[2025-05-25T10:27:41Z]     self.run()
[2025-05-25T10:27:41Z]   File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
[2025-05-25T10:27:41Z]     self._target(*self._args, **self._kwargs)
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 504, in run_engine_core
[2025-05-25T10:27:41Z]     raise e
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 491, in run_engine_core
[2025-05-25T10:27:41Z]     engine_core = EngineCoreProc(*args, **kwargs)
[2025-05-25T10:27:41Z]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 390, in __init__
[2025-05-25T10:27:41Z]     super().__init__(vllm_config, executor_class, log_stats,
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 71, in __init__
[2025-05-25T10:27:41Z]     self.model_executor = executor_class(vllm_config)
[2025-05-25T10:27:41Z]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/executor_base.py", line 52, in __init__
[2025-05-25T10:27:41Z]     self._init_executor()
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 46, in _init_executor
[2025-05-25T10:27:41Z]     self.collective_rpc("init_device")
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
[2025-05-25T10:27:41Z]     answer = run_method(self.driver_worker, method, args, kwargs)
[2025-05-25T10:27:41Z]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/utils.py", line 2605, in run_method
[2025-05-25T10:27:41Z]     return func(*args, **kwargs)
[2025-05-25T10:27:41Z]            ^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 604, in init_device
[2025-05-25T10:27:41Z]     self.worker.init_device()  # type: ignore
[2025-05-25T10:27:41Z]     ^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 144, in init_device
[2025-05-25T10:27:41Z]     self.model_runner: GPUModelRunner = GPUModelRunner(
[2025-05-25T10:27:41Z]                                         ^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 126, in __init__
[2025-05-25T10:27:41Z]     encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
[2025-05-25T10:27:41Z]                                                  ^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 94, in compute_encoder_budget
[2025-05-25T10:27:41Z]     ) = _compute_encoder_budget_multimodal(
[2025-05-25T10:27:41Z]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 124, in _compute_encoder_budget_multimodal
[2025-05-25T10:27:41Z]     .get_max_tokens_per_item_by_nonzero_modality(model_config)
[2025-05-25T10:27:41Z]      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 147, in get_max_tokens_per_item_by_nonzero_modality
[2025-05-25T10:27:41Z]     self.get_max_tokens_per_item_by_modality(model_config).items()
[2025-05-25T10:27:41Z]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 121, in get_max_tokens_per_item_by_modality
[2025-05-25T10:27:41Z]     return profiler.get_mm_max_tokens(
[2025-05-25T10:27:41Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 266, in get_mm_max_tokens
[2025-05-25T10:27:41Z]     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
[2025-05-25T10:27:41Z]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 179, in _get_dummy_mm_inputs
[2025-05-25T10:27:41Z]     return self.processor.apply(
[2025-05-25T10:27:41Z]            ^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1763, in apply
[2025-05-25T10:27:41Z]     ) = self._cached_apply_hf_processor(
[2025-05-25T10:27:41Z]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1529, in _cached_apply_hf_processor
[2025-05-25T10:27:41Z]     ) = self._apply_hf_processor_main(
[2025-05-25T10:27:41Z]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1372, in _apply_hf_processor_main
[2025-05-25T10:27:41Z]     prompt_ids = self._apply_hf_processor_text_only(prompt)
[2025-05-25T10:27:41Z]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1300, in _apply_hf_processor_text_only
[2025-05-25T10:27:41Z]     prompt_ids, _, _ = self._apply_hf_processor_text_mm(
[2025-05-25T10:27:41Z]                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1270, in _apply_hf_processor_text_mm
[2025-05-25T10:27:41Z]     processed_data = self._call_hf_processor(
[2025-05-25T10:27:41Z]                      ^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/aya_vision.py", line 181, in _call_hf_processor
[2025-05-25T10:27:41Z]     processed_outputs = super()._call_hf_processor(
[2025-05-25T10:27:41Z]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/processing.py", line 1233, in _call_hf_processor
[2025-05-25T10:27:41Z]     return self.info.ctx.call_hf_processor(
[2025-05-25T10:27:41Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2025-05-25T10:27:41Z]   File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 162, in call_hf_processor
[2025-05-25T10:27:41Z]     raise ValueError(msg) from exc
[2025-05-25T10:27:41Z] ValueError: Failed to apply AyaVisionProcessor on data={'text': '<image>'} with kwargs={'crop_to_patches': True}
[2025-05-25T10:27:42Z] [rank0]:[W525 03:27:41.939513420 ProcessGroupNCCL.cpp:1476] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

[zucchini-nlp]

Ah, I see, vLLM applies processor on text-only or mm-only data. So this might fail when the special tokens are not replaced by the processor. Since this is a no-op when the full processor pipe is ran, I will just remove it

Are there any other processor's failing because of the above PR? Our tests wouldn't catch errors related to calling processor on one modality only. I think it doesn't hurt for us to add one small test for that

[DarkLight1337]

This is the only processor that we found fails in this way.