python caching machine-learning streaming performance ai multi-node http2 load-balancer rpc asyncio ray model-serving gpu-monitoring inference-optimization llm llama-cpp ollama distributed-inference hybrid-routing