tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

skip some flash attn patches unless explicitly enabled (#643)

Browse Source 
* skip some flash attn patches if explicitly disabled

* make the other patches optional
This commit is contained in:
Wing Lian
2023-09-27 12:11:07 -04:00
committed by GitHub
parent e7d3e2dbb6
commit 895f0a0723
3 changed files with 38 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -636,6 +636,8 @@ flash_optimum:
xformers_attention:
# whether to use flash attention patch https://github.com/Dao-AILab/flash-attention:
flash_attention:
flash_attn_cross_entropy: # Whether to use flash-attention cross entropy implementation - advanced use only
flash_attn_rms_norm: # Whether to use flash-attention rms norm implementation - advanced use only
# whether to use scaled-dot-product attention
# https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html
sdp_attention: