Skip to content

Issues: flashinfer-ai/flashinfer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

57 Open 114 Closed
57 Open 114 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

C++ benchmarks CMake error caused by enable_fp16 option in generate.py
#734 opened Jan 13, 2025 by rtxxxpro
[RFC]: Introducing ReproSpec for Strong Reproducibility in LLM Inference
#733 opened Jan 11, 2025 by yzh119
ROPE problem in MLA decode kernel
#730 opened Jan 9, 2025 by lsw825
2
after torch compile with 0.2.0, speed is become very slow
#727 opened Jan 9, 2025 by MichoChan
6
Inconsistent results between different sequences with sequence lengths less than a single page size
#725 opened Jan 8, 2025 by fergusfinn
3
Custom mask slows down attention.
#724 opened Jan 8, 2025 by qiyuxinlin
2
How to use low bit KV Cache
#721 opened Jan 7, 2025 by sitabulaixizawaluduo
13
RuntimeError: Qwen2-VL does not support _Backend.FLASHINFER backend now
#720 opened Jan 7, 2025 by duzw9311
v0.1.6 release confusion
#715 opened Jan 3, 2025 by lflis
1
top_p_renorm_prob is not numerically stable for small probability values
#708 opened Dec 30, 2024 by merrymercy
@yzh119
[RFC] Un-fused softmax for short-query(decode) attention
#707 opened Dec 30, 2024 by yzh119
1
[RFC] Unifying AOT and JIT C++ Code via C Macros
#706 opened Dec 29, 2024 by yzh119
[Question] How to support custom stride of paged_kv for hopper prefill attention
#702 opened Dec 27, 2024 by jianfei-wangg
3
Fused-RoPE Attention with q_offset and k_offset
#701 opened Dec 26, 2024 by qiyuxinlin
1
Benchmarking MLA priority: high
#700 opened Dec 25, 2024 by YLGH
@yzh119
7
Different sequence numbers calculate inconsistent results
#696 opened Dec 24, 2024 by sitabulaixizawaluduo
@yzh119
10
Load the qwen2_5_insturcut_7b model, because the deployment request response using sglang is slower than that of vllm deployment. When troubleshooting, print("mutates_args",mutates_args,"schema_str",schema_str) is this result reasonable?
#695 opened Dec 24, 2024 by qingzhong1
1
Can't install latest flashinfer for torch 2.5
#692 opened Dec 23, 2024 by sleepwalker2017
@yzh119
Deprecation Notice: Python 3.8 Wheel Support to End in future releases
#682 opened Dec 18, 2024 by yzh119
2
[Roadmap] FlashInfer v0.2 to v0.3 roadmap
#675 opened Dec 17, 2024 by yzh119
15 tasks
[Bug] FlashInfer latest main wheel issue bug Something isn't working priority: high
#669 opened Dec 16, 2024 by zhyncs
4
Value error when running decode attention with FP8 KV cache
#661 opened Dec 13, 2024 by sugunav14
@yzh119
3
ValueError: The dtype of q torch.bfloat16 does not match the q_data_type torch.float16 specified in plan function.
#638 opened Nov 25, 2024 by Godlovecui
1
append_kv_cache's documentation is out of date
#616 opened Nov 19, 2024 by reyoung
@yzh119
1
[Question] Overflow risks when batch size and sequence length grows extremely large
#596 opened Nov 8, 2024 by rchardx
1
ProTip! Exclude everything labeled bug with -label:bug.