tocmo0nlord/axolotl

Files

History

NanoCode012 c78de6f214 feat: add kto example (#2158 ) [skip ci]

2024-12-09 08:17:27 -05:00

..

fft-8b-liger-fsdp.yaml

upgrade liger to 0.4.0 (#1973 )

2024-11-07 12:53:34 -05:00

fft-8b.yaml

add liger example (#1864 )

2024-08-23 12:37:50 -04:00

instruct-dpo-lora-8b.yml

examples: Fix config llama3 (#1833 ) [skip ci]

2024-10-14 16:00:48 -04:00

instruct-lora-8b.yml

examples: Fix config llama3 (#1833 ) [skip ci]

2024-10-14 16:00:48 -04:00

lora-1b-deduplicate-dpo.yml

Add Exact Deduplication Feature to Preprocessing Pipeline (#2072 )

2024-12-02 08:47:10 -05:00

lora-1b-deduplicate-sft.yml

Add Exact Deduplication Feature to Preprocessing Pipeline (#2072 )

2024-12-02 08:47:10 -05:00

lora-8b.yml

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 )

2024-07-19 00:47:07 -04:00

qlora-1b-kto.yaml

feat: add kto example (#2158 ) [skip ci]

2024-12-09 08:17:27 -05:00

qlora-1b.yml

memoize dataset length for eval sample packing (#1974 )

2024-10-17 15:15:29 -04:00

qlora-fsdp-70b.yaml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

qlora-fsdp-405b.yaml

qlora-fsdp ram efficient loading with hf trainer (#1791 )

2024-07-30 19:21:38 -04:00

qlora.yml

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 )

2024-07-19 00:47:07 -04:00

README.md

llama-3 examples (#1537 )

2024-04-18 14:28:03 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

Full Fine Tune
- Single GPU @ 48GB VRAM
LoRA
- Single GPU @ 11GB VRAM

QLORA+FSDP
- Dual GPU @ 21GB VRAM