Skip to content
#

multi-token-prediction

Star

Here are 2 public repositories matching this topic...

Language: Shell
Filter by language
All 12 Python 5 C++ 2 Shell 2 HTML 1 Jupyter Notebook 1

theogravity / dual-rtx-6000-blackwell-Gemma-4-31B-IT-NVFP4

Sponsor
Star 3

Optimized vLLM setup for Gemma 4 31B NVFP4 with MTP on dual RTX PRO 6000 Blackwell using vllm and docker: native FP4 Tensor Cores, Multi-Token Prediction (96.5% acceptance rate), and prefix caching. Includes benchmark results and replication scripts.

docker amd cuda gemma blackwell vllm llm-inference am5 speculative-decoding fp4 prefix-caching multi-token-prediction nvfp4 rtx-6000 gemma4 tensor-parallel
  • Updated May 10, 2026
  • Shell

Improve this page

Add a description, image, and links to the multi-token-prediction topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the multi-token-prediction topic, visit your repo's landing page and select "manage topics."

Learn more