bf9efe2a09
* [llama4] fix the mm yaml, add scout single gpu yaml * add README for llama4 * rename to specify fsdp
11 lines
412 B
Markdown
11 lines
412 B
Markdown
# Llama 4 by Meta AI
|
|
|
|
## Available Examples
|
|
|
|
### Llama 4 Scout 17Bx16Experts (109B)
|
|
- [Multi-Modal/Vision QLoRA w/ FSDP1](./scout-vision-qlora-fsdp.yaml)
|
|
- [Text Single GPU (H100) QLoRA](./scout-qlora-single-h100.yaml)
|
|
- [Text Multi GPU QLoRA w/ FSDP1](./scout-qlora-fsdp1.yaml)
|
|
|
|
Our Single GPU implementation for Llama 4 Scout uses only 68.5GB VRAM for post-training with 4k context length @ 546 tokens/second.
|