Files

DreamGenX 4d36ecc724 Sequential sample packing (#2404 ) [skip ci]

* add sequential sample packing

* chore: lint

---------

Co-authored-by: Wing Lian <wing@axolotl.ai>

2025-03-31 15:48:20 -04:00

fft-8b-liger-fsdp.yaml

feat: add config for optional parameters in a chat message (#2260 )

2025-02-18 09:59:27 +07:00

fft-8b.yaml

Add hub model id config options to all example yml files (#2196 ) [skip ci]

2024-12-17 11:24:30 -05:00

instruct-dpo-lora-8b.yml

feat: add config for optional parameters in a chat message (#2260 )

2025-02-18 09:59:27 +07:00

instruct-lora-8b.yml

feat: add config for optional parameters in a chat message (#2260 )

2025-02-18 09:59:27 +07:00

lora-1b-deduplicate-dpo.yml

feat: add config for optional parameters in a chat message (#2260 )

2025-02-18 09:59:27 +07:00

lora-1b-deduplicate-sft.yml

fix: minor patches for multimodal (#2441 )

2025-03-31 13:40:12 +07:00

lora-1b-kernels.yml

Activation function Triton kernels, LoRA custom autograd functions (#2324 )

2025-02-17 14:23:15 -05:00

lora-1b-ray.yml

Ray Train Axolotl Integration (#2251 )

2025-01-29 00:10:19 -05:00

lora-1b-sample-packing-sequentially.yml

Sequential sample packing (#2404 ) [skip ci]

2025-03-31 15:48:20 -04:00

lora-1b.yml

Add hub model id config options to all example yml files (#2196 ) [skip ci]

2024-12-17 11:24:30 -05:00

lora-8b.yml

Add hub model id config options to all example yml files (#2196 ) [skip ci]

2024-12-17 11:24:30 -05:00

qlora-1b-kto.yaml

Fixing KTO+QLoRA+multi-GPU (#2420 )

2025-03-21 10:18:28 -04:00

qlora-1b.yml

Add hub model id config options to all example yml files (#2196 ) [skip ci]

2024-12-17 11:24:30 -05:00

qlora-fsdp-70b.yaml

updating to fused (#2293 )

2025-01-30 11:45:56 -05:00

qlora-fsdp-405b.yaml

updating to fused (#2293 )

2025-01-30 11:45:56 -05:00

qlora.yml

Add hub model id config options to all example yml files (#2196 ) [skip ci]

2024-12-17 11:24:30 -05:00

README.md

llama-3 examples (#1537 )

2024-04-18 14:28:03 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

8B Base Model

Full Fine Tune
- Single GPU @ 48GB VRAM
LoRA
- Single GPU @ 11GB VRAM

70B Base Model

QLORA+FSDP
- Dual GPU @ 21GB VRAM