lioZ129 / context-parallelism Public

forked from malaysia-ai/context-parallelism

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Context Parallelism using Flex Attention, support Ring Attention.

0 stars 5 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 27 Commits
benchmark		benchmark
context_parallelism		context_parallelism
playground		playground
test		test
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
requirements.txt		requirements.txt
setup.py		setup.py

Repository files navigation

context-parallelism

Support Ring Attention https://arxiv.org/abs/2310.01889 using Flex Attention.

About

Context Parallelism using Flex Attention, support Ring Attention.

Report repository

Releases

No releases published

Packages

No packages published

Languages

Jupyter Notebook 76.7%
Python 23.3%