Repository navigation
Please also build docker for arm64 for cloud deployments #2057
Description
Hi, this is a feature request.
We tried running MLflow Models with Kserve and ran into a variety of errors. Turned out mlserver was the issue - currently we run all workloads on AWS EC2 instances with graviton processors (arm64).
The mlserver image pulled by kserve was not compatible.
Since a lot of cloud providers offer compute resources cheaper on their own arm64 chips; I think it would be great if mlsever would support arm64 as well.
Steps we took to solve it for now:
- clone this repository.
- Replace Dockerfile line 66 with
wget "https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERSION}/Miniforge3-${MINIFORGE_VERSION}-Linux-aarch64.sh" \ - ran
export DOCKER_DEFAULT_PLATFORM="linux/arm64/v8" - build the image
- overwrite the default kserve mlserver image with our newly compiled image
Since then everything works as expected; so I'd assume this might be a small feature request.
Activity
sakoush commentedon Feb 21, 2025
@DataCerealz This seems sensible thanks for pointing it out. Are you willing to contribute the suggested change as a PR please? We encourage contributions from the community.
DataCerealz commentedon Feb 24, 2025
Thanks for the fast response! I would love to contribute and hope to get to it this week. It seems like a straight forward enough fix since I have already done it with a fork. The only thing that might take me a bit more time is to test the image properly. I originally got a warning for a faulty
LD_LIBRARY_PATHduring build which I ignored since CUDA wasn't relevant for our use case anyway; but for a proper pull request everything should work.