Sunny Liu
1d7aee0ad2
ADOPT optimizer integration (#2032) [skip ci]
* adopt integration
* stuff
* doc and test for ADOPT
* rearrangement
* fixed formatting
* hacking pre-commit
* chore: lint
* update module doc for adopt optimizer
* remove un-necessary example yaml for adopt optimizer
* skip test adopt if torch<2.5.1
* formatting
* use version.parse
* specifies required torch version for adopt_adamw
---------
Co-authored-by: sunny <sunnyliu19981005@gmail.com>
Co-authored-by: Wing Lian <wing@axolotl.ai>