Files

Faria Huq 230e0ac363 Fix Lora config error for Llama3 (#1659 )

The current yml code throws an error: ValueError: Please set lora_modules_to_save to [`embed_tokens`, `lm_head`] when using an adapter and changing the special tokens.

I added the required changes to resolve it

2024-05-28 11:25:08 -04:00

fft-8b.yaml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

instruct-lora-8b.yml

Generalizing the chat_template prompt strategy (#1660 ) [skip ci]

2024-05-28 11:24:13 -04:00

lora-8b.yml

Fix Lora config error for Llama3 (#1659 )

2024-05-28 11:25:08 -04:00

qlora-fsdp-70b.yaml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

qlora.yml

update outputs path so that we can mount workspace to /workspace/data (#1623 )

2024-05-15 12:44:13 -04:00

README.md

llama-3 examples (#1537 )

2024-04-18 14:28:03 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

8B Base Model

Full Fine Tune
- Single GPU @ 48GB VRAM
LoRA
- Single GPU @ 11GB VRAM

70B Base Model

QLORA+FSDP
- Dual GPU @ 21GB VRAM