diff --git a/examples/llama-4/README.md b/examples/llama-4/README.md new file mode 100644 index 000000000..9ac987e49 --- /dev/null +++ b/examples/llama-4/README.md @@ -0,0 +1,10 @@ +# Llama 4 by Meta AI + +## Available Examples + +### Llama 4 Scout 17Bx16Experts (109B) +- [Multi-Modal/Vision QLoRA w/ FSDP1](./scout-vision-qlora.yaml) +- [Text Single GPU (H100) QLoRA](./scout-qlora-single-h100.yaml) +- [Text Multi GPU QLoRA w/ FSDP1](./scout-qlora-fsdp1.yaml) + +Our Single GPU implementation for Llama 4 Scout uses only 68.5GB VRAM for post-training with 4k context length @ 546 tokens/second. diff --git a/examples/llama-4/multimodal/scout-lora.yaml b/examples/llama-4/scout-vision-qlora.yaml similarity index 100% rename from examples/llama-4/multimodal/scout-lora.yaml rename to examples/llama-4/scout-vision-qlora.yaml