Pre-stage model weights with a ModelCache primitive

[dennis-upbound]

Fixes #66.

Serving pods fetched model weights from HuggingFace at every boot. For frontier models that's tens to hundreds of GB pulled on every pod start, and in a multi-node LeaderWorkerSet gang each pod pulls independently — so an 8-pod gang downloads the same 600GB eight times before it can serve. There was no way to stage weights once and share them.

This adds ModelCache, a v0.1 primitive that pre-stages a HuggingFace model onto a per-cluster ReadWriteMany PVC via a one-shot hydration Job, so serving pods mount the weights instead of downloading them. A deployment opts in with spec.modelCacheRef; the cache PVC mounts at /mnt/models on every serving pod, shared across the whole gang, and the weights are downloaded once per cluster and read N times.

The compose-model-cache function fans a ModelCache out to every matched InferenceCluster: a RWX PVC sized from the source, plus a hydration Job that runs hf download into it. Hydration is re-run safe via a completion marker, so an interrupted download resumes rather than serving truncated weights. Per-cluster phase (Pending/Hydrating/Ready/Failed) and an x/y ready summary are reported on the cache's status.

modelCacheRef now threads from ModelDeployment through to each ModelReplica, and the native and llm-d backends mount the cache. The mount is engine-agnostic — it lands on the native Deployment pod and on both the leader and worker templates of an llm-d gang. --model=/mnt/models is injected only for the turnkey vLLM path; a bring-your-own engine like SGLang sets its own --model-path, so injection is skipped when the engine supplies its own command.

On GKE the modelplane-rwx storage class self-provisions: compose-gke-cluster enables the Filestore API and compose-inference-cluster composes a VPC-pinned Filestore StorageClass. On EKS the cache works against an admin-provided EFS StorageClass (modelplane-rwx-efs); auto-provisioning EFS is a separate follow-up.

Before, a deployment had to bake the model into the engine args and re-pull every boot:

spec:
  workers:
    template:
      spec:
        containers:
        - name: engine
          args: ["--model=Qwen/Qwen3-0.6B"]   # fetched from HF on every pod start

After, the cache is staged once and mounted:

spec:
  modelCacheRef:
    name: qwen
  workers:
    template:
      spec:
        containers:
        - name: engine
          args: ["--model=/mnt/models"]       # read from the mounted PVC

Scope is locked to the HuggingFace source plus a Modelplane-managed PVC, matching the merged XRD. EKS works against an admin-provided EFS StorageClass; auto-provisioning EFS is a separate follow-up.

How it was validated so far: unit tests for every changed function (the cache function's PVC/Job/status/conditions, the deployment→replica propagation, and the backend mounts including the vLLM-inject vs SGLang-skip split). Live cluster validation is pending.

I have:

• Read and followed Modelplane's contribution process.
• Run nix flake check and made sure it passes. Per-function unit tests pass and ruff lint/format is clean locally; the full sandboxed nix flake check runs in CI on this PR.
• Added or updated tests covering the composition function changes.
• Signed off every commit with git commit -s.

[negz]

Thanks @dennis-upbound!

really like the PR description. I found it way easier to read. Did you get an agent to generate it using the new guidance in CONTRIBUTING.md?

[dennis-upbound]

Thanks @dennis-upbound!

really like the PR description. I found it way easier to read. Did you get an agent to generate it using the new guidance in CONTRIBUTING.md?

yep! thanks for setting up the guidance. I had to still clean a little llm fluff but overall not bad