Wing Lian 68f1b7004c ScatterMoE LoRA support (#3410) ...

* scattermoe lora support

* fsdp, bf16, dim fixes

* expert weights aren't needed in save for bwd since they are frozen

* use sonicmoe optim options

* update save model from upstream

* fixes per code review feedback and add tests

* revert removal of CP fix

* misc fixes

2026-02-24 14:59:55 -05:00

12 KiB

Raw Blame History

View Raw