NanoCode012 6a8baf8fa7 feat: add sonicmoe (#3411) ...

* feat: add sonicmoe

* feat: add torch compile for routing

* feat: add routing smoke test

* feat: add qwen3_5_moe, qwen3_vl_moe, qwen3_omni_moe

* fix: disable mlp kernel for sonicmoe too

* feat: update to sonicmoe release

* chore: update import following new sonicmoe changes

* feat: update handling for blackwell

* feat: add sonicmoe e2e test

* fix: installation for updated sonicmoe

* fix: git commit

* fix: ignore py req and fix metadata

* fix: increase min hidden size to match sonicmoe kernel min

* fix: attempt properly interleave and handle unpatch mid-test

* chore: refactor teardown better

* chore: refactor to re-use rearrange

* fix: add idempotency guard

* fix: address comments on CI memory and interleave

* fix: tests grad, param doublewrapped

2026-03-05 13:43:31 -05:00

..

integrations

feat: add sonicmoe (#3411)

2026-03-05 13:43:31 -05:00

kernels

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

multigpu

mark slow tests that are timing out in CI (#3428) [skip ci]

2026-03-02 12:26:30 -05:00

patched

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

solo

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

__init__.py

missing dunder-init

2023-11-06 18:33:01 -05:00

.gitignore

Support Sample packing for phi arch (#586)

2023-09-15 15:46:54 -04:00

test_activation_offloading.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_deepseekv3.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_diffusion.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_dpo.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_embeddings_lr.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_evaluate.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_falcon.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_gemma2.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_gemma3_text.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_imports.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_llama_pretrain.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_llama_vision.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_llama.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_load_model.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_lora_llama.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_mamba.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_mistral.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_mixtral.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_optimizers.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_packing_loss.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_phi.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_preprocess.py

Fix typos in some files (#3250) [skip ci]

2025-11-07 08:21:20 -05:00

test_process_reward_model_smollm2.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_profiler.py

allow profiling in mid-training rather from the start (#2899) [skip ci]

2025-07-14 20:11:11 -04:00

test_qat.py

add: support mxfp4 axo (#3375)

2026-03-05 13:40:45 -05:00

test_quantization.py

add: support mxfp4 axo (#3375)

2026-03-05 13:40:45 -05:00

test_qwen.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_reward_model_smollm2.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_save_first_step.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00

test_schedulers.py

Add ruff, remove black, isort, flake8, pylint (#3092)

2025-08-23 23:37:33 -04:00

test_streaming.py

Changes from dataset_processes to dataset_num_proc (#3352) [skip ci]

2026-02-10 17:44:17 +07:00

test_tokenizer.py

add tokenizer_save_jinja_files to keep legacy behavior of including chat template in tokenizer_config.json (#3093)

2025-08-26 09:30:04 -04:00

utils.py

transformers v5 upgrade (#3272)

2026-01-27 17:08:24 -05:00