tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Merge pull request #92 from OpenAccess-AI-Collective/flash-optimum

Browse Source 
add support for opimum bettertransformers
This commit is contained in:
Wing Lian
2023-06-14 07:50:15 -04:00
committed by GitHub
parent 06674a11f2 fd2c9814c9
commit 16bb6276a5
11 changed files with 367 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -421,6 +421,8 @@ optimizer:
# specify weight decay
weight_decay:
# whether to bettertransformers
flash_optimum:
# whether to use xformers attention patch https://github.com/facebookresearch/xformers:
xformers_attention:
# whether to use flash attention patch https://github.com/HazyResearch/flash-attention: