Attention Implementation in vLLM

[llsj14]

• attention backend docs: docs
• backend selection logic: vllm.attention.selector.py API
• adding a custom attention backend
  • should implement custom backend under the vllm.attention.backends module and register it in the selector logic
  • #7315 (FlexAttention)
  • #5083 (Triton-only backend)
  • PR #21366 (Auto-Select Attention Backend)

[llsj14]

The most critical parts in the vLLM codebase for attention backend integration are:

1. The vllm.attention.selector module, especially the get_attn_backend function, which determines and loads the appropriate backend based on model and environment. This is where you would add logic to recognize and select your custom backend.
2. The vllm.attention.backends.abstract module, which defines the AttentionBackend interface. Your custom backend should inherit from this and implement the required methods.
3. The vllm.attention.layer.Attention class, which acts as the main entry point for attention computation and delegates to the selected backend.
   For extensibility, see also PR #12218 (passing the attention object to the backend) and PR #10558 (unified attention interface registration), which show how the system is designed for pluggable backends.
   Would you like a step-by-step guide for adding a new backend?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!