add muon optimizer

optimizer_cls_and_kwargs is on trainer_kwargs only add adamw_kwargs if they're non-null fix mocks better handling of override and check the optimizer unwrap optimizer
2025-02-27 23:15:41 -05:00
parent 0134093acc
commit 0542c7dd56
10 changed files with 296 additions and 158 deletions
--- a/requirements.txt
+++ b/requirements.txt
@@ -63,3 +63,4 @@ torchao==0.7.0
 schedulefree==1.3.0

 axolotl-contribs-lgpl==0.0.3
+axolotl-contribs-mit==0.0.3