Can the docker images be made smaller?

[mcteo]

Hi all,

The intelanalytics/ipex-llm-inference-cpp-xpu docker image is ~27GB when unpacked, which is huge, and preventing me from deploying iGPU accelerated Ollama on my server.

There's a few obviously redundancies:

• There's two versions of drivers installed.
• Development packages like git, sudo, vim, less are installed.
• ~4gb of nvidia-* python packages are installed (a transitive dependency of accelerate).

Would it be possible to provide more granular docker images, with these redundancies remove?

Thanks

[liu-shaojun]

Hi, thanks for the feedback!

The base image we use is based on Intel’s oneAPI, which includes a comprehensive runtime and development environment for Intel GPUs. As a result, the image is relatively large (~27GB unpacked), but it's necessary to ensure compatibility and performance.

If you're looking for a smaller and more lightweight deployment method, we recommend using our portable zip-based packages, which avoid many of the dependencies present in the full Docker image:

• llamacpp portable zip quickstart
• Ollama portable zip quickstart

These options should be much easier to deploy.

[Ellie-Williams-007]

Yeah this could be great to be smaller!