tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

set zero3 optimizer betas to auto so they inherit from HF trainer config (#507)

Browse Source
This commit is contained in:
Aman Gupta Karmani
2023-08-30 08:10:33 -04:00
committed by GitHub
parent 76576323df
commit 1e07c162f1
1 changed files with 1 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
deepspeed/zero3.json
View File

@@ -35,10 +35,7 @@
"type": "AdamW", "type": "AdamW",
"params": { "params": {
"lr": "auto", "lr": "auto",
"betas": [ "betas": "auto",
0.9,
0.95
],
"eps": 1e-8, "eps": 1e-8,
"weight_decay": "auto" "weight_decay": "auto"
} }