tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

set auto for other params that hf trainer sets for ds. include zero1 json (#570)

Browse Source
This commit is contained in:
Wing Lian
2023-09-14 11:04:37 -04:00
committed by GitHub
parent 3fbde762ab
commit 3b18c963cc
3 changed files with 42 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
deepspeed/zero3.json
View File

@@ -36,7 +36,7 @@
"params": {
"lr": "auto",
"betas": "auto",
"eps": 1e-8,
"eps": "auto",
"weight_decay": "auto"
}
},