Principal Engineer, AI/ML. Twenty-one years building production systems — the last several on generative AI (RAG platforms, agentic workflows, evaluation and governance), and lately on the layer underneath them: inference serving.
Current focus — serving-layer economics. Batch size buys cost amortization; utilization costs tail latency. They are different knobs with different curves, and most of the interesting engineering lives in the tension between them: prefix caching, KV-cache management, class-aware scheduling, admission control at a measured goodput knee, and the instrumentation that proves any of it still works next month.
A control plane over vLLM — session-affinity routing, EDF-plus-floor scheduling, admission control — plus a from-scratch mini-engine that rebuilds paged KV blocks, hash-chain prefix caching and continuous batching in ~600 lines of PyTorch so the mechanisms are readable instead of vendored.
Includes a measured ablation: 5× TTFT improvement from prefix caching, with a flat control line, on two consumer GPUs. Five annotated sequence-diagram traces walk one request through admission, cache match, prefill and decode.
Retrieval evaluation and release gating (golden sets, regression-relative thresholds, drift detection) · multi-tenant RAG architecture · streaming data platforms (Kafka, Flink, Spark, CDC) · AI governance and guardrails for regulated, learner-facing products · scaling engineering practices from a small core team to a cross-functional org.