Skip to content
#

streaming-inference-engine

Star

Here are 2 public repositories matching this topic...

Language: All
Filter by language
All 2 JavaScript 1 Python 1

flatseek / flatrun

Star 0

A complete inference runtime for open-weight large language models, enabling efficient execution through streaming weights, quantization, and memory-aware scheduling

machine-learning deep-learning numpy inference quantization inference-engine memory-mapping inference-optimization kv-cache ai-inference cpu-inference large-language-models llm model-runtime safetensors llm-inference gguf streaming-inference gguf-runner streaming-inference-engine
  • Updated Aug 1, 2026
  • Python

Improve this page

Add a description, image, and links to the streaming-inference-engine topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the streaming-inference-engine topic, visit your repo's landing page and select "manage topics."

Learn more