Dan Saunders 23f0c51d88 Sequence parallelism (#2412) ...

* adding easy_context as integration for now

* progress on ring attn impl

* progress on ring attn impl

* cleanup

* remove errant file

* fix req

* removing unused code

* updates

* pytest

* update

* updates

* fixes

* precommit fixes

* working multi-group SP

* fixing sample packing

* remove debug logs and simplify

* eval dataloader and sampler changes

* removing some obvious comments

* update config.qmd and rename option

* scoping down problematic import

* another import scoping change

* pernicious Fire CLI bugfix

* isolate cli tests

* actually isolate CLI tests

* gracefully handle no ring-flash-attn

* fix

* fix

* move ring flash attn to extras with flash-attn (#2414)

* removing flash-attn from requirements.txt (in setup.py extras already)

* rename file, delete another

* using field validator instead of model validator

* test fix

* sampler / dataloader refactor

* non-seq2se1 collator fix

* removing print statement

* bugfix

* add SP doc, review comments

* small changes

* review comments, docstrings

* refactors, SP mixin

* small updates

* fix tests

* precommit

* precommit

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>
Co-authored-by: Dan Saunders <dan@axolotl.ai>

2025-03-21 12:43:55 -04:00

..

integrations

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

kernels

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

multigpu

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

patched

Sequence parallelism (#2412)

2025-03-21 12:43:55 -04:00

solo

bump HF versions except for trl (#2427)

2025-03-20 10:22:05 -04:00

__init__.py

missing dunder-init

2023-11-06 18:33:01 -05:00

.gitignore

Support Sample packing for phi arch (#586)

2023-09-15 15:46:54 -04:00

test_deepseekv3.py

feat: add deepseek_v3 sample packing (#2230)

2025-02-24 15:03:15 -05:00

test_dpo.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_embeddings_lr.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_falcon.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_imports.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00

test_llama_pretrain.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_llama_vision.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_llama.py

pass additional info for fix untrained tokens when using distributed + offloading (#2388)

2025-03-11 12:02:43 -04:00

test_load_model.py

updating to fused (#2293)

2025-01-30 11:45:56 -05:00

test_lora_llama.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_mamba.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_mistral.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_mixtral.py

Optimizer refactor and add Muon support (#2367)

2025-03-06 11:49:19 -05:00

test_optimizers.py

Optimizer refactor and add Muon support (#2367)

2025-03-06 11:49:19 -05:00

test_packing_loss.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_phi.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_process_reward_model_smollm2.py

Misc fixes 20250130 (#2301)

2025-01-31 08:58:04 -05:00

test_qwen.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_reward_model_smollm2.py

feat: add config for optional parameters in a chat message (#2260)

2025-02-18 09:59:27 +07:00

test_schedulers.py

Add REX LR Scheduler (#2380)

2025-03-05 10:26:11 -05:00

utils.py

adding pre-commit auto-update GH action and bumping plugin versions (#2428)

2025-03-21 11:02:43 -04:00