[NeurIPS'23] Speculative Decoding with Big Little Decoder
-
Updated
Feb 6, 2024 - Python
#
speculative-execution
Here are 6 public repositories matching this topic...
Run AI models too large for your Mac's memory — at near-full speed. Intelligent expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon.
python macos rust machine-learning inference moe quantization mlx speculative-execution mixture-of-experts memory-optimization apple-silicon llm metal-gpu ssd-streaming expert-caching
-
Updated
May 11, 2026 - Python
State-aware hedged requests for serverless GPU inference — return the first valid result and cancel the losers with an audited receipt.
python modal inference cold-start speculative-execution mlops serverless-gpu runpod hedged-requests cerebrium
-
Updated
Jul 16, 2026 - Python
Run large MLX models on Apple Silicon with flash weight streaming, using native precision beyond RAM limits
menubar clustering optimization transformer quantization speculative-execution swiftui bpe memory-optimization apple-silicon bpe-encoder llm-training ssd-streaming weight-streaming expert-caching
-
Updated
Jul 25, 2026 - Python
Two-tier speculative intent architecture — fires on partial input, reconciles with LLM in parallel. Patent pending.
nvidia speculative-execution intent-recognition llama-cpp faster-whisper patent-pending edge-ai-llm-voice-assistant
-
Updated
May 28, 2026 - Python
Extension of the ScheduleFlow Simulator to allow speculative request times at submission and during backfill
-
Updated
Jul 28, 2020 - Python
Improve this page
Add a description, image, and links to the speculative-execution topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the speculative-execution topic, visit your repo's landing page and select "manage topics."