Use CPU-only PyTorch to reduce Docker image size#977
Open
r33drichards wants to merge 3 commits intomainfrom
Open
Use CPU-only PyTorch to reduce Docker image size#977r33drichards wants to merge 3 commits intomainfrom
r33drichards wants to merge 3 commits intomainfrom
Conversation
Add UV_EXTRA_INDEX_URL environment variable to download PyTorch from the CPU-only wheel index, reducing image size by ~1.5GB by excluding unnecessary CUDA libraries. https://claude.ai/code/session_01JCxwLh9kRhD7jW7SqzfbhJ
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The PyTorch CPU wheel index contains older versions of packages like requests that conflict with other dependencies. Using unsafe-best-match allows uv to pick the best compatible version from any index. https://claude.ai/code/session_01JCxwLh9kRhD7jW7SqzfbhJ
- Add flake.nix that builds a base image with Python 3.12 and uv - Use CPU-only PyTorch via UV_EXTRA_INDEX_URL for smaller image size - Update workflow to use Nix + podman for multi-arch builds - Use skopeo for pushing images to ECR - Build natively on arm64 and amd64 runners (no cross-compilation) - Remove Dockerfile in favor of Nix-based image building https://claude.ai/code/session_01JCxwLh9kRhD7jW7SqzfbhJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimizes the Docker image for the docs-mcp-server by configuring PyTorch to use CPU-only wheels instead of the default CUDA-enabled build, reducing image size by approximately 1.5GB.
Changes
UV_EXTRA_INDEX_URLenvironment variable pointing to PyTorch's CPU wheel indexDetails
The docs-mcp-server doesn't require GPU acceleration, so including CUDA libraries in the PyTorch installation is unnecessary overhead. By configuring the package manager to use CPU-only wheels from PyTorch's official index, we significantly reduce the final image size without sacrificing functionality.
This change improves build times and reduces storage/deployment costs while maintaining full compatibility with the application's requirements.
https://claude.ai/code/session_01JCxwLh9kRhD7jW7SqzfbhJ