Skip to content
#

overload

Here are 5 public repositories matching this topic...

Simulates LLM serving admission control strategies (None, MaxConcurrency, TokenBudget, SLOAware, Predictive) across 135 configurations. Key findings: tight token budget (4096) achieves 10x better goodput than loose budget; predictive control is the only strategy maintaining SLO compliance at 100ms; SLOAware is fundamentally unstable under sustained

  • Updated Jul 10, 2026
  • Python

Improve this page

Add a description, image, and links to the overload topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the overload topic, visit your repo's landing page and select "manage topics."

Learn more