Pytorch implementation of Ring Attention and Striped Ring Attention.
Ring attention is a distributed attention operator that compute attention across devices.
./run_single.sh
-
Notifications
You must be signed in to change notification settings - Fork 1
66RING/ring-attention-pytorch
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
tiny ring attention implement for learning purpose
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published