You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
unload_model answers loaded:false while the LLMInferenceService remains #242
unload_model answered "loaded": false for qwen3-5-4b-hf (backend kserve), but the LLMInferenceService stayed in place and Ready, with its predictor pod running. A delete_node_pool right after it was refused because the pool still served that model:
node pool <pool> still runs 1 busy node(s): node <node> runs model-serving/qwen3-5-4b-hf-kserve-… (1 GPU),
serving 1 model(s): LLMInferenceService model-serving/qwen3-5-4b-hf (Qwen/Qwen3.5-4B) — unload them first
kubectl get llminferenceservice -n model-serving then showed qwen3-5-4b-hf Ready, aged 9m48s: it was the object the preceding load_model had created, so it had never been deleted. On the same pool, an earlier unload_model of the same model removed it as expected, so the failure is intermittent. Observed on an AWS test installation, no model cache configured. The answer carried inventory.reason: "no cache claim is configured; there is no inventory to rescan".
Proposed solution
Have unload_model derive loaded from the serving objects after its delete: the LLMInferenceService gone, or carrying a deletionTimestamp, answered as status: Terminating. A delete that was not issued or that failed must surface as an error, never as loaded: false.
Acceptance criteria
unload_model answers loaded: false only when the LLMInferenceService is gone or terminating, and otherwise an error naming the object.
A test covers an unload whose delete does not take effect.
Fixed in #277, released as v1.14.1-rc.1. unload_model now reads every LLMInferenceService back after its delete: gone or terminating is an unload; still there without a deletionTimestamp is a conflict naming the object, and the model stays served and wired. Proof: TestStopRefusesWhenTheDeleteDoesNotTakeEffect and TestStopAnswersATerminatingObjectAsStopped, make test green, release artifacts published.
Activity
teemow commentedon Oct 8, 2026
Fixed in #277, released as v1.14.1-rc.1.
unload_modelnow reads every LLMInferenceService back after its delete: gone or terminating is an unload; still there without a deletionTimestamp is aconflictnaming the object, and the model stays served and wired. Proof:TestStopRefusesWhenTheDeleteDoesNotTakeEffectandTestStopAnswersATerminatingObjectAsStopped,make testgreen, release artifacts published.Written by an agent.