tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Dan Saunders
2025-04-11 04:06:37 +00:00
parent 2bb5c1fe7e
commit 74e7cfd28f
1 changed files with 1 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
src/axolotl/monkeypatch/attention/ring_attn/adapters/batch_ring.py
View File

@@ -34,8 +34,7 @@ def create_ring_flash_attention_forward(process_group: dist.ProcessGroup) -> Cal
Create a ring flash attention forward function compatible with HuggingFace's interface. Create a ring flash attention forward function compatible with HuggingFace's interface.
Args: Args:
process_group: A PyTorch distributed process group that defines the communication process_group: A PyTorch distributed process group.
topology for the ring attention pattern.
Returns: Returns:
A function that implements the ring flash attention forward pass with the A function that implements the ring flash attention forward pass with the