llama-3 examples (#1537)

2024-04-18 14:28:03 -04:00
parent d28ba2e405
commit 15f7910d33
4 changed files with 217 additions and 0 deletions
--- a/examples/llama-3/README.md
+++ b/examples/llama-3/README.md
@@ -0,0 +1,13 @@
+# Llama-3
+
+https://llama.meta.com/llama3/
+
+[8B Base Model](https://huggingface.co/meta-llama/Meta-Llama-3-8B)
+ - [Full Fine Tune](./fft-8b.yaml)
+   - Single GPU @ 48GB VRAM
+ - [LoRA](./lora-8b.yml)
+   - Single GPU @ 11GB VRAM
+
+[70B Base Model](https://huggingface.co/meta-llama/Meta-Llama-3-70B)
+ - [QLORA+FSDP](./qlora-fsdp-70b.yaml)
+   - Dual GPU @ 21GB VRAM