tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122) [skip ci]

Browse Source 
* set fp16 to false if bf16, update bf16: auto in example YAMLs

* unset fp16 so that it fallsback properly if bf16 isn't available

* Update README.md [skip-ci]

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>

* test that bf16 disables fp16

---------

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
This commit is contained in:
Wing Lian
2024-01-22 18:44:01 -05:00
committed by GitHub
parent eaaeefce55
commit 782b6a4216
38 changed files with 86 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/pythia/lora.yml
View File

@@ -27,7 +27,7 @@ num_epochs: 4
learning_rate: 0.00001
train_on_inputs: false
group_by_length: false
bf16: true
bf16: auto
tf32: true
early_stopping_patience:
resume_from_checkpoint: