Run large language models on hardware that normally can't fit them.
Sleeper is a local inference engine focused on a single problem: making models that normally require datacenter GPUs or multi-card servers runnable on a single consumer or workstation machine — privately, and without renting a cluster.
For now, this repo is the home for Sleeper's public benchmarks — more may follow later. The goal of the project is to close the gap between "this model is too big for my machine" and "I need to run it anyway."
- Runs models that are substantially larger than the available GPU memory, on a single machine.
- Keeps everything local — no cloud, no external API, full control over your data and prompts.
- Prioritizes being able to run the model at all on constrained hardware, accepting a speed trade-off in exchange.
- Works with popular open-weight model families.
- Not a cloud service or hosted API.
- Not dependent on the whole model fitting into GPU memory.
- Not tied to specialized or exotic hardware beyond a standard GPU.
Sleeper is proprietary and under active development. Only published benchmark evidence lives here — the implementation and internal documentation are not part of this repo.