Support Ring Attention https://arxiv.org/abs/2310.01889 and Tree Attention https://arxiv.org/abs/2408.04093 using Flex Attention.
mesolitica/context-parallelism
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Support Ring Attention https://arxiv.org/abs/2310.01889 and Tree Attention https://arxiv.org/abs/2408.04093 using Flex Attention.