[Bug] AudioQnA failed with multilang on xeon after vllm version update to v0.9.0.1

[chensuyue]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-SPR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other
• N/A

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Kubernetes GMC
• Other
• N/A

Running nodes

Single Node

What's the version?

#1921

Description

Update vLLM version to v0.9.0.1, AudioQnA failed with multilang test on xeon.
https://github.com/opea-project/GenAIExamples/actions/runs/15525520248/job/43709327677?pr=1921

Reproduce steps

bash test_compose_multilang_on_xeon.sh

Raw log

ERROR:    Exception in ASGI application
  Traceback (most recent call last):
    File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
      result = await app(  # type: ignore[func-returns-value]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
      return await self.app(scope, receive, send)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
      await super().__call__(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__
      await self.middleware_stack(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
      raise exc
    File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
      await self.app(scope, receive, _send)
    File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 177, in __call__
      raise exc
    File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 175, in __call__
      await self.app(scope, receive, send_wrapper)
    File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
      await self.app(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
      await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      raise exc
    File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
      await app(scope, receive, sender)
    File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 714, in __call__
      await self.middleware_stack(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 734, in app
      await route.handle(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
      await self.app(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
      await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      raise exc
    File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
      await app(scope, receive, sender)
    File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 73, in app
      response = await f(request)
                 ^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
      raw_response = await run_endpoint_function(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
      return await dependant.call(**values)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/audioqna_multilang.py", line 109, in handle_request
      result_dict, runtime_graph = await self.megaservice.schedule(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/comps/cores/telemetry/opea_telemetry.py", line 61, in wrapper
      res = await func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/comps/cores/mega/orchestrator.py", line 151, in schedule
      response, node = await done_task
                       ^^^^^^^^^^^^^^^
    File "/home/user/comps/cores/telemetry/opea_telemetry.py", line 61, in wrapper
      res = await func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/comps/cores/mega/orchestrator.py", line 370, in execute
      response = await session.post(endpoint, json=input_data)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 768, in _request
      resp = await handler(req)
             ^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 746, in _connect_and_send_request
      await resp.start(conn)
    File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 528, in start
      message, payload = await protocol.read()  # type: ignore[union-attr]
                         ^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 672, in read
      await self._waiter
  aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
  base64: invalid input
  Error: Process completed with exit code 1.

Attachments

No response

[yinghu5]

Hi @chensuyue

may be investigate the PR: #2061 0.9.1?

Try the bash test_compose_multilang_on_xeon.sh today, it seems work fine now. (vllm is vllm 0.9.0.1+cpu)

and also try the compose's vllm image , the vllm seems up to 0.9.0.2.dev0+g5fbbfe9a4.d20250613.cpu and the Audio example runs fine.

[yinghu5]

wait for vllm next release maybe 0.9.2