GLM-5.2-NVFP4-REAP-469B serving on SM120 (4× RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Attention + MTP speculative decode
-
Updated
Jun 19, 2026 - Shell
#
sm120
Here are 16 public repositories matching this topic...
From-scratch C++/CUDA inference engine for the NVIDIA RTX 5090 (sm_120a) — the best single-GPU backend for agentic AI: tool calling, long-context loops, reasoning and concurrent sub-agents on top of the fastest single-stream decode on the 5090 (beats llama.cpp, at-or-ahead of vLLM on NVFP4). 100% written by Claude Code.
cpp cuda quantization inference-engine mixture-of-experts blackwell llm safetensors llama-cpp vllm local-llm llm-inference qwen gguf fp4 rtx-5090 sm120 nvfp4 rtx-pro-6000 gated-deltanet
-
Updated
Jun 26, 2026 - Cuda
NVFP4 inference on Blackwell GeForce (RTX 5090/5080/5070 Ti/RTX PRO 6000) — SM120 patches for vLLM + FlashInfer + CUTLASS. 175 tok/s on Qwen3.6-35B MoE.
gpu-computing quantization cutlass gemm geforce blackwell vllm llm-inference flashinfer rtx-5090 sm120 nvfp4
-
Updated
Apr 27, 2026 - Python
Reproducible recipe: serve abliterated Gemma-4-12B (gemma4_unified) at 50-118 tok/s on no-NVLink Blackwell (SM120) via vLLM nightly + ModelOpt FP8/NVFP4 + MTP spec-decode.
quantization gemma blackwell fp8 vllm speculative-decoding sm120 nvfp4 abliterated gemma-4 modelopt tensor-parallel
-
Updated
Jun 7, 2026 - Python
Lna-Lab production pipeline: GGUF -> modelopt-format NVFP4 + working MTP head for vLLM on RTX PRO 6000 Blackwell (SM120). Stages 2 (NVFP4) and 3 (MTP graft) are Lna-Lab originals; stage 1 (GGUF->bf16) reuses li-yifei/gguf-to-nvfp4.
-
Updated
Apr 27, 2026 - Python
Rust-native MoE inference runtime with custom CUDA kernels for Blackwell GPUs. Includes DFlash speculative decoding, multi-tier Engram memory, and entropy-adaptive routing. Targets Qwen3.5-35B-A3B on a single RTX 5060 Ti 16GB.
rust ffi cuda inference moe quantization mamba state-space-models deltanet blackwell engram llm qwen speculative-decoding sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Apr 25, 2026 - Rust
Optimized vLLM deployment for NVIDIA Blackwell (RTX 5090) on Linux Kernel 6.14. Resolves SM_120 kernel incompatibilities, P2P deadlocks, and memory fragmentation for high-performance LLM inference.
-
Updated
Jan 17, 2026 - Dockerfile
Production-grade FlashAttention FP8 e4m3 forward kernel for NVIDIA Blackwell consumer GPUs (sm_120a, e.g. RTX PRO 6000). 647–652 TFLOPS at hd=128, sl=8192. Multi-kernel dispatcher, C library with Go and Python bindings
cuda transformer attention gpu-kernels tensor-cores blackwell fp8 flash-attention fp8e4m3 sm120 rtx-pro-6000 sm-120a
-
Updated
Jun 12, 2026 - Cuda
Downstream llama.cpp TurboQuant CUDA fork with adaptive KV layout selection for long-context inference on consumer Blackwell GPUs.
cuda inference moe quantization blackwell kv-cache long-context llama-cpp local-llm rtx-5080 sm120 turboquant
-
Updated
May 1, 2026 - C++
Pre-built onnxruntime-gpu 1.24.1 with Blackwell sm_120 CUDA kernels (RTX 5090/5080/5070)
machine-learning gpu cuda nvidia python-wheel onnxruntime blackwell rtx-5070 rtx-5090 rtx-5080 sm120
-
Updated
Feb 14, 2026
llama.cpp fork with additional SOTA quants and improved performance
cuda inference llama cuda-kernels quantization ssm mamba state-space-models blackwell llama-cpp gguf sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Apr 26, 2026 - C++
Complete installation guide for ComfyUI-Hunyuan3DWrapper on NVIDIA Blackwell GPUs (RTX 5070 Ti, 5080, 5090) Covers custom_rasterizer manual compilation for sm_120 / compute_120 architecture.
windows cuda cuda-toolkit cuda-support windows-11 windows-package blackwell comfyui hunyuan3d hunyuan3dwrapper blackwell-gpu sm120 rtx-5070-ti
-
Updated
Apr 12, 2026
Experimental xFormers + MSLK builds validated on NVIDIA Blackwell (SM120) GPUs, CUDA 12.8, PyTorch 2.11, and RTX 5070 Laptop hardware.
windows cuda pytorch nvidia attention blackwell stable-diffusion xformers automatic1111 stable-diffusion-webui a1111 sm120 rtx5070 rtx50-series
-
Updated
Jun 17, 2026 - Python
Serve an abliterated Gemma-4-12B at high speeds on Blackwell GPUs without NVLink using vLLM, FP8 quantization, and MTP speculative decoding.
quantization gemma blackwell fp8 vllm speculative-decoding sm120 nvfp4 abliterated modelopt tensor-parallel
-
Updated
Jun 27, 2026 - Python
Improve this page
Add a description, image, and links to the sm120 topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sm120 topic, visit your repo's landing page and select "manage topics."