2 Commits

Author	SHA1	Message	Date
Wing Lian	efa1209a92	add smoke test training	2024-10-30 15:40:27 -04:00
Wing Lian	78e12f8ca5	add basic support for the optimi adamw optimizer (#1727) ... * add support for optimi_adamw optimizer w kahan summation * pydantic validator for optimi_adamw * workaround for setting optimizer for fsdp * make sure to install optimizer packages * make sure to have parity for model parameters passed to optimizer * add smoke test for optimi_adamw optimizer * don't use foreach optimi by default	2024-07-14 19:12:57 -04:00