NanoCode012 842fa039dd feat: add sonicmoe fused lora support (#3519) ...

* feat: add sonicmoe fused lora support

* fix: forgot to add file

* feat: add test

* feat: add lora support for other routes

* fix: add int8 lora support

* fix: add qwen35_moe interleave support

* fix: qwen3_5_moe loss

* chore: lint

* address some pr comments

* fix test imports

* add support matrix for moe kernels [skip ci]

---------

Co-authored-by: Wing Lian <wing@axolotl.ai>

2026-04-02 08:53:48 -04:00

10 KiB

Raw Blame History

View Raw