Files

Dan Saunders 6f6d917a99 Revert "checkpoint model on first step callback (#2906 )"

This reverts commit 10ba1622f7.

2025-07-15 15:01:12 -04:00

3b-qat-fsdp2.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

fft-8b-liger-fsdp.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

fft-8b.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

instruct-dpo-lora-8b.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

instruct-lora-8b.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-1b-deduplicate-dpo.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-1b-deduplicate-sft.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-1b-kernels.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-1b-ray.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-1b-sample-packing-sequentially.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-1b.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

lora-8b.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

qlora-1b-kto.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

qlora-1b.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

qlora-fsdp-70b.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

qlora-fsdp-405b.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

qlora.yml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

README.md

llama-3 examples (#1537 )

2024-04-18 14:28:03 -04:00

sparse-finetuning.yaml

Revert "checkpoint model on first step callback (#2906 )"

2025-07-15 15:01:12 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

8B Base Model

Full Fine Tune
- Single GPU @ 48GB VRAM
LoRA
- Single GPU @ 11GB VRAM

70B Base Model

QLORA+FSDP
- Dual GPU @ 21GB VRAM