Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
-
Updated
Aug 15, 2025 - Python
#
sparse-attention
Here are 37 public repositories matching this topic...
Trainable fast and memory-efficient sparse attention
-
Updated
May 28, 2026 - Python
[ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention
-
Updated
Mar 6, 2026 - Python
[NeurIPS 2025] Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation
-
Updated
Nov 11, 2025 - Python
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention
transformer video-generation mlsys inference-acceleration ai-infra linear-attention sparse-attention diffusion-transformer train-acceleration sparse-linear-attention
-
Updated
Feb 24, 2026 - Python
[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
-
Updated
May 1, 2025 - Python
Efficient triton implementation of Native Sparse Attention.
-
Updated
May 23, 2025 - Python
Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
-
Updated
Oct 13, 2025 - Python
[CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
-
Updated
Jan 14, 2026 - Python
[ICLR 2026] SparseD: Sparse Attention for Diffusion Language Models
-
Updated
Feb 22, 2026 - Python
Advancing the frontier of efficient AI
-
Updated
May 15, 2026 - Python
Vortex: A Flexible and Efficient Sparse Attention Framework
-
Updated
May 27, 2026 - Python
[TIP-2025] Official Pytorch implementation of "Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution"
-
Updated
Jul 8, 2025 - Python
Official repository for "SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space"
-
Updated
May 7, 2026 - Python
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
attention-mechanism fast-inference sparse-neural-networks low-rank vision-transformer efficient-transformers sparse-attention efficient-vision-transformers
-
Updated
Jul 4, 2023 - Python
Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extension on a single H200 | by RightNow https://www.rightnowai.co/
cuda triton attention sparse-attention long-context deepseek deepseek-v4 compressed-sparse-attention lightning-indexer
-
Updated
May 5, 2026 - Python
Dynamic Attention Mask (DAM) generate adaptive sparse attention masks per layer and head for Transformer models, enabling long-context inference with lower compute and memory overhead without fine-tuning.
-
Updated
Jun 16, 2025 - Python
The code implementation of paper "VORTA: Efficient Video Diffusion via Routing Sparse Attention"
-
Updated
Oct 15, 2025 - Python
From-scratch reimplementation of DeepSeek's Native Sparse Attention (arXiv:2502.11089) in Triton + CUDA Hopper WGMMA. 7.4x faster than FlashAttention-3 at 64k context. Five-model training fleet, perplexity sweep, LongBench v2, MoBA comparison.
cuda pytorch transformer triton hopper attention-mechanism nsa gpu-kernels sparse-attention llm long-context flash-attention deepseek native-sparse-attention wgmma
-
Updated
May 12, 2026 - Python
From-scratch, paper-faithful PyTorch implementation of DeepSeek-V4 architecture for transparent study, testing, ablation, and mini-scale training.
deep-learning transformers pytorch language-model from-scratch mixture-of-experts research-implementation sparse-attention llm long-context deepseek deepseek-v4 compressed-attention
-
Updated
May 26, 2026 - Python
Improve this page
Add a description, image, and links to the sparse-attention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sparse-attention topic, visit your repo's landing page and select "manage topics."