[llama4] fix the mm yaml, add scout single gpu yaml (#2510)

* [llama4] fix the mm yaml, add scout single gpu yaml

* add README for llama4

* rename to specify fsdp
This commit is contained in:
Wing Lian
2025-04-09 02:52:45 -04:00
committed by GitHub
parent 0dac2ddeac
commit bf9efe2a09
3 changed files with 133 additions and 23 deletions

View File

@@ -0,0 +1,10 @@
# Llama 4 by Meta AI
## Available Examples
### Llama 4 Scout 17Bx16Experts (109B)
- [Multi-Modal/Vision QLoRA w/ FSDP1](./scout-vision-qlora-fsdp.yaml)
- [Text Single GPU (H100) QLoRA](./scout-qlora-single-h100.yaml)
- [Text Multi GPU QLoRA w/ FSDP1](./scout-qlora-fsdp1.yaml)
Our Single GPU implementation for Llama 4 Scout uses only 68.5GB VRAM for post-training with 4k context length @ 546 tokens/second.