Skip to content
#

24gb-vram

Star

Here is 1 public repository matching this topic...

Language: Python
Filter by language
All 2 HTML 1 Python 1

DominikBucko / qwen38-flash-next-3090

Star 10

Run Qwen3.8-Flash-Next on ONE RTX 3090 (24 GB) + 64 GB RAM: 128K context, up to 2,100 tok/s prefill, 43–51 tok/s decode. vLLM runtime with hot MoE experts on the GPU and cold experts computed on the CPU, INT8 KV cache, Docker, OpenAI-compatible API.

moe mixture-of-experts single-gpu rtx3090 int4 llm long-context vllm local-llm llm-inference qwen kv-cache-quantization openai-compatible-api consumer-gpu rtx-3090 cpu-offloading qwen38 qwen3-8-flash-next 128k-context 24gb-vram
  • Updated Sep 30, 2026
  • Python

Add this topic to your repo

To associate your repository with the 24gb-vram topic, visit your repo's landing page and select "manage topics."

Learn more