Skip to content

Issues: Dao-AILab/flash-attention

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

562 Open 527 Closed
562 Open 527 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Question about disabling the causal mask
#1304 opened Oct 30, 2024 by volcverse
Error in line 21 O_i adjustment in Algorithm 1 in FlashAttention-3 Paper
#1303 opened Oct 30, 2024 by hasanunlu
whl for torch 2.5.0
#1302 opened Oct 28, 2024 by Galaxy-Husky
1
flash_attn_with_kvcach return block_lse or attention_score
#1301 opened Oct 28, 2024 by NonvolatileMemory
2
FlashSelfAttention and SelfAttention in flash_attn.modules.mha give different results
#1300 opened Oct 28, 2024 by senxiu-puleya
5
using out argument will change the output
#1299 opened Oct 27, 2024 by youkaichao
Build stuck on torch2.5.0
#1295 opened Oct 23, 2024 by ycformal
7
any plan for varlen fwd support hopper FP8?
#1294 opened Oct 22, 2024 by pengwu22
Request for New Release with PT Compile Ops
#1293 opened Oct 22, 2024 by kostum123
Support for CUDA 12.4 and above? URGENT PERHAPS?
#1292 opened Oct 22, 2024 by BBC-Esq
7
Support different shape attention mask
#1291 opened Oct 22, 2024 by SunzeY
CUTLASS 3.5.1 makes Flash Attention 3 slower?
#1289 opened Oct 22, 2024 by fno2010
4
undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE
#1287 opened Oct 20, 2024 by LanXingXuan
In unit test，how is the dropout_fraction diff tolerance selected?
#1286 opened Oct 18, 2024 by muoshuosha
How to profile standard attention written in pytorch?
#1283 opened Oct 18, 2024 by woongjoonchoi
FlashAttention installation error: "CUDA 11.6 and above" requirement issue
#1282 opened Oct 17, 2024 by 21X5122
Softcap for FlashAttention v3
#1281 opened Oct 16, 2024 by Jeff-Zilence
1
ImportError: cannot import name 'flash_attn_unpadded_qkvpacked_func' from 'flash_attn.flash_attn_interface, why i cannot import it?? QAQ
#1280 opened Oct 16, 2024 by YANGTUOMAO
Unable to import my new kernel function after compilation success.
#1278 opened Oct 15, 2024 by jpli02
2
Why does the flash_attn_varlen_func method increase GPU memory usage?
#1277 opened Oct 15, 2024 by shaonan1993
1
Is there a way to install flash-attention without specific cuda version ?
#1276 opened Oct 14, 2024 by HuangChiEn
Concurrent Warp Group Execution in FA3: Tensor Core Resource Limitation?
#1275 opened Oct 13, 2024 by ziyuhuang123
Does FA2 support 4D attention mask?
#1274 opened Oct 12, 2024 by XiangTodayEatsWhat
Six Flash-Attention-3 unit tests fail on H20
#1272 opened Oct 11, 2024 by cailun01
4
Would using both strategies simultaneously theoretically result in better overlap between TC and MUFU? How could this be explained with a diagram?
#1271 opened Oct 11, 2024 by ziyuhuang123
ProTip! Type g i on any issue or pull request to go back to the issue listing page.