tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Feat: Add warmup_ratio (#893)

Browse Source 
* Feat: Add warmup_ratio

* fix: update readme with more details on conflict
This commit is contained in:
NanoCode012
2023-11-25 12:15:43 +09:00
committed by GitHub
parent 9fc29e082b
commit fb12895a17
4 changed files with 43 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@@ -675,7 +675,8 @@ gradient_accumulation_steps: 1
micro_batch_size: 2
eval_batch_size:
num_epochs: 4
warmup_steps: 100
warmup_steps: 100 # cannot use with warmup_ratio
warmup_ratio: 0.05 # cannot use with warmup_steps
learning_rate: 0.00003
lr_quadratic_warmup:
logging_steps: