Dan Saunders 80304c26a7 SP GRPO support + batch SP fixes (#2643) ...

* ctx manager for SP

* updates

* update

* further simplifying

* simplifying

* simplifying

* reorg

* batch api HF adapter for ring-flash-attn; cleanup and improvements

* update

* adding all batch ring-flash-attn methods via single adapter

* fix

* fixes for batch API funcs, simplify

* fix

* grpo sp support

* progress

* stronger subclassing of TRL GRPO trainer; custom distributed sampler

* subclassing constructor

* progress

* finalizing SP + GRPO trainer

* minimize diffs to GRPO trainer

* remove (most of) the custom GRPO trainer logic

* debug

* debug

* update

* update

* update

* progress

* cleanup

* cleanup

* minor changes

* update

* update

* update

* small changes

* updates

* cleanup; torch.compile ring_flash_attn functions to prevent numerical instability; lint

* spacing

* cleanup; log in pydantic model config only on main process

* remove comment

* fix sp sampler, update to latest upstream code, doc

* add docs

* update quartodoc autodoc contents

* fix, simplifications

* fixes + simplifications

* review comments

* lint

* removing main process only logs in favor of #2608

* fixes, additional smoke test

* updatse

* more tests

* update

* fix grad accum bug (sort of)

* lint, tests

* todo

2025-05-12 17:52:40 -04:00

..

dataset-formats

feat(doc): add split_thinking docs (#2613) [skip ci]

2025-05-06 20:05:32 -04:00

images

Ray Train Axolotl Integration (#2251)

2025-01-29 00:10:19 -05:00

.gitignore

Autodoc generation with quartodoc (#2419)

2025-03-21 12:26:47 -04:00

amd_hpc.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

batch_vs_grad.qmd

Feat: update doc (#1475) [skip ci]

2024-04-04 13:43:40 +09:00

cli.qmd

Fix(doc): add delinearize instruction (#2545)

2025-04-24 01:03:43 -04:00

config.qmd

update doc and use P2P=LOC for brittle grpo test (#2649)

2025-05-12 14:17:25 -04:00

custom_integrations.qmd

Add: Sparse Finetuning Integration with llmcompressor (#2479)

2025-05-01 12:25:16 -04:00

dataset_loading.qmd

Feat: Add doc on loading datasets and support for Azure/OCI (#2482)

2025-04-07 12:41:13 -04:00

dataset_preprocessing.qmd

Autodoc generation with quartodoc (#2419)

2025-03-21 12:26:47 -04:00

debugging.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

docker.qmd

chore(doc): update docker tags on doc (#2559) [skip ci]

2025-04-25 17:14:48 -04:00

faq.qmd

feat: add eos_tokens and train_on_eot for chat_template EOT parsing (#2364)

2025-04-28 10:11:20 -04:00

fsdp_qlora.qmd

github urls (#1734)

2024-07-11 09:19:29 -04:00

getting-started.qmd

Feat: minor docs improvements for RLHF and faq on embeddings (#2401) [skip ci]

2025-03-17 08:39:04 -04:00

inference.qmd

Feat: minor docs improvements for RLHF and faq on embeddings (#2401) [skip ci]

2025-03-17 08:39:04 -04:00

input_output.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

installation.qmd

Fix(doc): add delinearize instruction (#2545)

2025-04-24 01:03:43 -04:00

lora_optims.qmd

feat: add support for multimodal in lora kernels (#2472) [skip ci]

2025-04-02 09:33:46 -04:00

lr_groups.qmd

support for custom lr groups for non-embedding modules (#2213)

2025-01-24 12:56:28 -05:00

mac.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

multi-gpu.qmd

feat(doc): explain deepspeed configs (#2514) [skip ci]

2025-04-11 09:52:43 -04:00

multi-node.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

multimodal.qmd

fix(doc): key used to point to url in multimodal doc (#2575) [skip ci]

2025-04-29 15:10:59 -04:00

multipack.qmd

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

nccl.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

ray-integration.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

reward_modelling.qmd

chore(docs): add cookbook/blog link to docs (#2410) [skip ci]

2025-03-17 08:38:19 -04:00

rlhf.qmd

fix(doc): clarify vllm usage with grpo (#2573) [skip ci]

2025-04-28 10:07:45 -04:00

sequence_parallelism.qmd

SP GRPO support + batch SP fixes (#2643)

2025-05-12 17:52:40 -04:00

torchao.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00

unsloth.qmd

Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)

2025-02-25 16:09:37 +07:00