Disaggregate prefill and decode: prefill block, role-aware scheduler, and InferencePool routing#124

Closed

dennis-upbound wants to merge 11 commits into

mainfrom

dennis/disagg-impl

Commits on Jun 11, 2026

Disaggregate prefill and decode onto co-located pod sets

dennis-upbound
and
claude
committed
Verify the llm-d EPP mechanism and Envoy AI Gateway for Phase 2 routing

dennis-upbound
and
claude
committed
Require explicit counts and routing on disaggregated deployments

dennis-upbound
and
claude
committed
Label disaggregated pods for the llm-d endpoint picker

dennis-upbound
and
claude
committed
Pin the Phase 2 EPP deploy model, gateway addon values, and images
dennis-upbound
committed
Inject the pd-sidecar on disaggregated decode pods

dennis-upbound
and
claude
committed
Front disaggregated decode with a GAIE InferencePool

dennis-upbound
and
claude
committed
Emit the llm-d endpoint picker for disaggregated replicas
dennis-upbound
committed
Install the Envoy AI Gateway stack and GAIE CRDs on serving clusters
dennis-upbound
committed
Satisfy the formatter
dennis-upbound
committed
Fix InferencePool failureMode placement and the llm-d backend docstring

dennis-upbound
and
claude
committed