fix(docker): CPU inference broken by unpinned torch; root-owned ~/.cache#945
fix(docker): CPU inference broken by unpinned torch; root-owned ~/.cache#945thiagomargatho wants to merge 2 commits into
Conversation
Unpinned torch resolves to the latest release (2.13 at the time of writing), which breaks CPU inference in every engine tested: - Kokoro/LuxTTS: "Cannot copy out of meta tensor; no data!" - Qwen3-TTS: "unsupported scalarType" inside torch.autocast 2.7.1 is the newest version that works and matches hume-tada's declared range (torch>=2.7,<2.8). Pinned in the Dockerfile's CPU branch rather than requirements.txt so ROCm builds keep resolving torch from the ROCm wheel index untouched. Qwen3-TTS is now installed with --no-deps (like chatterbox-tts and hume-tada already are) so its dependency list can't re-resolve torch/transformers and undo the pin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Docker creates /home/voicebox/.cache (parent of the huggingface-cache volume mountpoint) owned by root on every container create, and a fresh named volume is root-owned too. The app runs as the voicebox user (uid 999), so anything that writes a cache outside the HF mount (torch hub, spacy, ...) fails with Permission denied. The entrypoint already runs as root before dropping privileges via gosu — create/chown the cache dirs there (non-recursive, instant). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe Dockerfile adds pinned CPU PyTorch installation and preserves those versions during later package installs. The ROCm entrypoint creates the Hugging Face cache directory and assigns it to the application user. ChangesContainer runtime updates
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Dockerfile`:
- Around line 58-65: Update the CPU installation command in the Dockerfile’s
torch/torchaudio fallback branch to use the PyTorch CPU wheel index at
https://download.pytorch.org/whl/cpu while preserving the existing pinned
versions and installation prefix.
In `@scripts/rocm-entrypoint.sh`:
- Around line 15-20: Update the cache ownership setup around mkdir and chown to
recursively repair ownership of all existing descendants under
/home/voicebox/.cache/huggingface, while retaining ownership correction for the
parent cache directory. Ensure reused volumes with root-owned files and
subdirectories become writable by voicebox.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: df8a33a9-33cf-4357-8eb3-3fb8244c6677
📒 Files selected for processing (2)
Dockerfilescripts/rocm-entrypoint.sh
| else \ | ||
| pip install --no-cache-dir --prefix=/install torch==2.7.1 torchaudio==2.7.1; \ | ||
| fi | ||
| # ^ CPU builds pin torch: 2.8+ breaks CPU inference in every engine tested | ||
| # (Kokoro/LuxTTS: "Cannot copy out of meta tensor"; Qwen: "unsupported | ||
| # scalarType" in torch.autocast). 2.7.1 is the newest that works and matches | ||
| # hume-tada's torch>=2.7,<2.8. Installed before requirements.txt so the open | ||
| # torch range there keeps this version instead of resolving to latest. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
set -euo pipefail
echo '--- Dockerfile context ---'
nl -ba Dockerfile | sed -n '45,80p'
echo
echo '--- Search for torch install patterns ---'
rg -n "torch==|torchaudio==|download.pytorch.org|cpu" Dockerfile requirements*.txt pyproject.toml . || trueRepository: jamiepine/voicebox
Length of output: 223
🏁 Script executed:
set -euo pipefail
echo '--- Dockerfile context ---'
sed -n '45,80p' Dockerfile | cat -n
echo
echo '--- Search for torch install patterns ---'
rg -n "torch==|torchaudio==|download.pytorch.org|cpu" Dockerfile requirements*.txt pyproject.toml . || trueRepository: jamiepine/voicebox
Length of output: 50374
🌐 Web query:
PyTorch 2.7.1 pip install default index CPU wheel torch torchaudio documentation
💡 Result:
To install the CPU-only version of PyTorch 2.7.1, you should use the official PyTorch wheel index to ensure the correct CPU-only binaries are fetched [1][2]. Use the following pip command: pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cpu Key details for this installation: 1. Index URL: It is important to use the --index-url flag with https://download.pytorch.org/whl/cpu. Using --find-links (-f) or omitting the index URL can lead to unintended installations of CUDA-enabled dependencies [2]. 2. Version Compatibility: The command above includes the matching versions of torchvision and torchaudio that were typically released alongside PyTorch 2.7.1 [1]. 3. Official Documentation: While PyTorch 2.13 is the current version as of July 2026 [3][4], specific installation instructions for historical versions like 2.7.1 are maintained in the official Previous PyTorch Versions documentation [1]. You can access general documentation for PyTorch at https://docs.pytorch.org/docs/stable/index.html [5].
Citations:
- 1: https://pytorch.org/get-started/previous-versions/
- 2: Why torch CPU version installing CUDA libraries? pytorch/pytorch#159560
- 3: https://github.com/pytorch/pytorch/releases/tag/v2.13.0
- 4: https://pytorch.org/blog/pytorch-2-13-release-blog/
- 5: https://docs.pytorch.org/docs/stable/index.html
Use the CPU wheel index here.
torch==2.7.1 / torchaudio==2.7.1 for the CPU image should come from https://download.pytorch.org/whl/cpu; the plain default index can resolve the wrong wheel set.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Dockerfile` around lines 58 - 65, Update the CPU installation command in the
Dockerfile’s torch/torchaudio fallback branch to use the PyTorch CPU wheel index
at https://download.pytorch.org/whl/cpu while preserving the existing pinned
versions and installation prefix.
| # Docker creates the HF volume mountpoint's parent (~/.cache) as root on every | ||
| # container create, and a fresh volume is root-owned too — without this the app | ||
| # user can't write any cache (torch, spacy) outside the HF mount. | ||
| mkdir -p /home/voicebox/.cache/huggingface | ||
| chown voicebox:voicebox /home/voicebox/.cache /home/voicebox/.cache/huggingface | ||
|
|
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Fix ownership of existing cache contents.
The persistent huggingface-cache volume can already contain root-owned hub subdirectories and files. This non-recursive chown changes only the two directory entries, so voicebox may still fail when updating an existing cache after an upgrade or reused volume. Apply a one-time recursive ownership migration, or otherwise repair all existing descendants.
Proposed fix
mkdir -p /home/voicebox/.cache/huggingface
-chown voicebox:voicebox /home/voicebox/.cache /home/voicebox/.cache/huggingface
+chown -R voicebox:voicebox /home/voicebox/.cache📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Docker creates the HF volume mountpoint's parent (~/.cache) as root on every | |
| # container create, and a fresh volume is root-owned too — without this the app | |
| # user can't write any cache (torch, spacy) outside the HF mount. | |
| mkdir -p /home/voicebox/.cache/huggingface | |
| chown voicebox:voicebox /home/voicebox/.cache /home/voicebox/.cache/huggingface | |
| # Docker creates the HF volume mountpoint's parent (~/.cache) as root on every | |
| # container create, and a fresh volume is root-owned too — without this the app | |
| # user can't write any cache (torch, spacy) outside the HF mount. | |
| mkdir -p /home/voicebox/.cache/huggingface | |
| chown -R voicebox:voicebox /home/voicebox/.cache |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/rocm-entrypoint.sh` around lines 15 - 20, Update the cache ownership
setup around mkdir and chown to recursively repair ownership of all existing
descendants under /home/voicebox/.cache/huggingface, while retaining ownership
correction for the parent cache directory. Ensure reused volumes with root-owned
files and subdirectories become writable by voicebox.
1 participant
Summary
Two independent Docker fixes, found while running the CPU image in production for ~a week:
1. CPU builds: pin torch 2.7.1 (unpinned torch → 2.13 → CPU inference broken)
With
torch>=2.2.0unpinned, pip resolves the latest release. On torch 2.13 CPU inference fails in every engine we tested:Cannot copy out of meta tensor; no data!unsupported scalarTypeinsidetorch.autocast2.7.1 is the newest version that works, and it matches hume-tada's declared range (
torch>=2.7,<2.8— currently unenforced since it's installed with--no-deps).The pin lives in the Dockerfile's CPU branch (the
elseof the existing ROCm conditional), not inrequirements.txt, so ROCm builds keep resolving torch from the ROCm wheel index for bothROCM_VERSION=6.3and7.2— a hard pin in requirements would have no matching wheel on the 7.2 index.Qwen3-TTS is now installed with
--no-deps(same as chatterbox-tts and hume-tada) so its dependency list can't re-resolve torch/transformers and silently undo the pin.2. Entrypoint: chown
~/.cache(root-owned on every container create)Docker creates
/home/voicebox/.cache— the parent of thehuggingface-cachevolume mountpoint — as root on every container create, and a fresh named volume is root-owned too. The app runs asvoicebox(uid 999), so any cache write outside the HF mount (torch hub, spacy, …) fails withPermission denied, and on a fresh volume even HF model downloads fail.The entrypoint already runs as root before
gosu, so a non-recursivemkdir -p+chownthere fixes both cases at zero cost.Testing
~/.cacheand the HF mountpoint come up owned byvoicebox, writable by uid 999.gosu).🤖 Generated with Claude Code
Summary by CodeRabbit