Files

RodriMora 219cd0d3c5 Fix eval_sample_packing in llama-3 lora example (#1716 ) [skip ci]

* Fix eval_sample_packing in llama-3 lora example

* Update examples/llama-3/lora-8b.yml

Co-authored-by: Wing Lian <wing.lian@gmail.com>

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-07-13 14:34:44 -04:00

fft-8b.yaml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

instruct-lora-8b.yml

Generalizing the chat_template prompt strategy (#1660 ) [skip ci]

2024-05-28 11:24:13 -04:00

lora-8b.yml

Fix eval_sample_packing in llama-3 lora example (#1716 ) [skip ci]

2024-07-13 14:34:44 -04:00

qlora-fsdp-70b.yaml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

qlora.yml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

README.md

llama-3 examples (#1537 )

2024-04-18 14:28:03 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

8B Base Model

Full Fine Tune
- Single GPU @ 48GB VRAM
LoRA
- Single GPU @ 11GB VRAM

70B Base Model

QLORA+FSDP
- Dual GPU @ 21GB VRAM