tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Dion optimizer support (#3014)

Browse Source 
* Add support for Dion optimizer

* dion training kwargs

* fix var names

* no dion 8bit for now

* use updated axolotl-contribs-mit for dion optimizer

* add smoke test for dion optimizer

* add docs

* fix typo during edits

* fix test to not remove load in 8bit
This commit is contained in:
Wing Lian
2025-08-04 16:33:30 -04:00
committed by GitHub
parent 33d094721c
commit ab49d16e34
10 changed files with 145 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
_quarto.yml
View File

@@ -284,6 +284,7 @@ website:
- docs/sequence_parallelism.qmd
- docs/gradient_checkpointing.qmd
- docs/nd_parallelism.qmd
- docs/optimizers.qmd
- section: "Troubleshooting"
contents: