You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
JIT pre-compilation pass at startup. CUDA kernel pre-loading. Configurable WarmupOptions. Readiness probe gates on warm-up completion.
At server startup, run configurable dummy inference passes after model loading but before marking IsReady=true. This triggers .NET Tier-0→Tier-1 JIT promotion (with Dynamic PGO profile data) and exercises CUDA/cuBLAS kernel pipelines, eliminating the cold-start latency penalty on the first real request.
Acceptance Criteria
WarmupOptions sealed record in DotLLM.Engine with Enabled, DummyPrompt, MaxTokens, Iterations properties
WarmupRunner static class that executes warm-up passes using TextGenerator
Server integration: warm-up runs in ServerStartup.LoadModel() before IsReady=true
Hot-swap: warm-up runs automatically on model swap via LoadModel()
CLI flags: --no-warmup (disable), --warmup-iterations N on serve command
Per-iteration timing logged with [dotllm] prefix
/ready probe returns 503 during warm-up, 200 after completion
Activity
Step 46: Warm-up — JIT pre-compilation and CUDA kernel pre-loading (#92)
Address PR review feedback (#92)
Step 46: Warm-up — JIT pre-compilation, CUDA kernel pre-loading (#92) (…