xzuyn 0134093acc Add REX LR Scheduler (#2380) ...

* Update trainer_builder.py

* Update base.py

* Update __init__.py

* Update base.py

* Update base.py

* Update config.qmd

* Update base.py

* Update base.py

* Update base.py

* Update base.py

* Update base.py

* Update base.py

* Update base.py

* lint

* lint

* lint

* lint

* lint

* lint

* Update base.py

* Update base.py

* lint

* Update base.py

* Update base.py

* Move RexLR to `schedulers.py`

* Remove RexLR from `base.py`

* Fix tooltip formatting

* lint

* Create test_schedulers.py

* Use a default optimizer in test

* lint

* lint

* Add `warmup_steps` and `cosine_min_lr_ratio` to test

* lint

2025-03-05 10:26:11 -05:00

..

integrations

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

kernels

Activation function Triton kernels, LoRA custom autograd functions (#2324)

2025-02-17 14:23:15 -05:00

multigpu

GRPO (#2307)

2025-02-13 16:01:01 -05:00

patched

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

solo

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

__init__.py

missing dunder-init

2023-11-06 18:33:01 -05:00

.gitignore

Support Sample packing for phi arch (#586)

2023-09-15 15:46:54 -04:00

test_deepseekv3.py

feat: add deepseek_v3 sample packing (#2230)

2025-02-24 15:03:15 -05:00

test_dpo.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_embeddings_lr.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_falcon.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_imports.py

Bump deepspeed 20240727 (#1790)

2024-07-27 10:24:11 -04:00

test_llama_pretrain.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_llama_vision.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_llama.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_load_model.py

updating to fused (#2293)

2025-01-30 11:45:56 -05:00

test_lora_llama.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_mamba.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_mistral.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_mixtral.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_optimizers.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_packing_loss.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_phi.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_process_reward_model_smollm2.py

Misc fixes 20250130 (#2301)

2025-01-31 08:58:04 -05:00

test_qwen.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_reward_model_smollm2.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_schedulers.py

Add REX LR Scheduler (#2380)

2025-03-05 10:26:11 -05:00

utils.py

GRPO (#2307)

2025-02-13 16:01:01 -05:00