From 3036ca349f99396f758637ceb610b96803b5f867 Mon Sep 17 00:00:00 2001 From: Wing Lian Date: Wed, 9 Apr 2025 02:15:09 -0400 Subject: [PATCH] add README for llama4 --- examples/llama-4/README.md | 10 ++++++++++ .../scout-lora.yaml => scout-vision-qlora.yaml} | 0 2 files changed, 10 insertions(+) create mode 100644 examples/llama-4/README.md rename examples/llama-4/{multimodal/scout-lora.yaml => scout-vision-qlora.yaml} (100%) diff --git a/examples/llama-4/README.md b/examples/llama-4/README.md new file mode 100644 index 000000000..9ac987e49 --- /dev/null +++ b/examples/llama-4/README.md @@ -0,0 +1,10 @@ +# Llama 4 by Meta AI + +## Available Examples + +### Llama 4 Scout 17Bx16Experts (109B) +- [Multi-Modal/Vision QLoRA w/ FSDP1](./scout-vision-qlora.yaml) +- [Text Single GPU (H100) QLoRA](./scout-qlora-single-h100.yaml) +- [Text Multi GPU QLoRA w/ FSDP1](./scout-qlora-fsdp1.yaml) + +Our Single GPU implementation for Llama 4 Scout uses only 68.5GB VRAM for post-training with 4k context length @ 546 tokens/second. diff --git a/examples/llama-4/multimodal/scout-lora.yaml b/examples/llama-4/scout-vision-qlora.yaml similarity index 100% rename from examples/llama-4/multimodal/scout-lora.yaml rename to examples/llama-4/scout-vision-qlora.yaml