NanoCode012 842fa039dd feat: add sonicmoe fused lora support (#3519) ...

* feat: add sonicmoe fused lora support

* fix: forgot to add file

* feat: add test

* feat: add lora support for other routes

* fix: add int8 lora support

* fix: add qwen35_moe interleave support

* fix: qwen3_5_moe loss

* chore: lint

* address some pr comments

* fix test imports

* add support matrix for moe kernels [skip ci]

---------

Co-authored-by: Wing Lian <wing@axolotl.ai>

2026-04-02 08:53:48 -04:00

..

__init__.py

upgrade liger to 0.4.0 (#1973)

2024-11-07 12:53:34 -05:00

test_diffusion_callback.py

roundup_power2_divisions not needed with newer pytorch versions (#3540)

2026-03-24 15:40:05 -04:00

test_diffusion.py

roundup_power2_divisions not needed with newer pytorch versions (#3540)

2026-03-24 15:40:05 -04:00

test_kd_chat_template.py

fix: Fix evaluation loss in KD trainer (#3271)

2025-12-17 13:40:36 -05:00

test_liger.py

feat: add arg to enable dft in liger (#3125)

2025-11-10 21:37:47 +07:00

test_nemo_gym.py

Nemo gym integration (#3516) [skip ci]

2026-03-25 07:38:06 -04:00

test_routing_parity.py

feat: add sonicmoe fused lora support (#3519)

2026-04-02 08:53:48 -04:00

test_scattermoe_autotune_telemetry.py

fix for flaky tests in lora ops kernels w autotune (#3511) [skip ci]

2026-03-19 01:18:47 -04:00

test_scattermoe_lora_kernels.py

Scattermoe LoRA optimizations (#3513)

2026-03-19 23:07:42 -04:00

test_scattermoe_lora.py

consolidate behavioud of routing in scattermoe kernels (#3475)

2026-03-16 23:47:40 -04:00

test_sonicmoe_gradients.py

feat: add sonicmoe fused lora support (#3519)

2026-04-02 08:53:48 -04:00

test_sonicmoe_lora.py

feat: add sonicmoe fused lora support (#3519)

2026-04-02 08:53:48 -04:00

test_sonicmoe.py

feat: add sonicmoe fused lora support (#3519)

2026-04-02 08:53:48 -04:00

test_swanlab.py

roundup_power2_divisions not needed with newer pytorch versions (#3540)

2026-03-24 15:40:05 -04:00