Files
axolotl/README.md

383 lines
21 KiB
Markdown

<div align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/887513285d98132142bf5db2a74eb5e0928787f1/image/axolotl_logo_digital_white.svg">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/887513285d98132142bf5db2a74eb5e0928787f1/image/axolotl_logo_digital_black.svg">
<img alt="Axolotl" src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/887513285d98132142bf5db2a74eb5e0928787f1/image/axolotl_logo_digital_black.svg" width="400" height="104" style="max-width: 100%; margin-bottom: 20px;">
</picture>
<h3 style="color: #2563eb; font-weight: 600; margin: 20px 0 10px 0;">🧠 Streamlined AI Model Post-Training</h3>
<p style="font-size: 18px; color: #6b7280; margin-bottom: 30px; max-width: 600px;">
A powerful, flexible tool designed to streamline post-training for various AI models with enterprise-grade features and optimizations.
</p>
</div>
<div align="center" style="margin: 30px 0;">
<table style="border: none; margin: 0 auto;">
<tr>
<td style="border: none; padding: 5px;">
<img src="https://img.shields.io/github/license/axolotl-ai-cloud/axolotl.svg?color=blue&style=for-the-badge" alt="GitHub License">
</td>
<td style="border: none; padding: 5px;">
<img src="https://img.shields.io/github/stars/axolotl-ai-cloud/axolotl?style=for-the-badge&color=yellow" alt="GitHub Repo stars">
</td>
</tr>
<tr>
<td style="border: none; padding: 5px;">
<img src="https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/tests.yml/badge.svg?style=for-the-badge" alt="tests">
</td>
<td style="border: none; padding: 5px;">
<a href="https://codecov.io/gh/axolotl-ai-cloud/axolotl">
<img src="https://img.shields.io/codecov/c/github/axolotl-ai-cloud/axolotl?style=for-the-badge" alt="codecov">
</a>
</td>
</tr>
</table>
</div>
<div align="center" style="margin: 20px 0;">
<a href="https://github.com/axolotl-ai-cloud/axolotl/graphs/contributors" style="text-decoration: none; margin: 0 10px;">
<img src="https://img.shields.io/github/contributors-anon/axolotl-ai-cloud/axolotl?color=green&style=for-the-badge&logo=github" alt="contributors">
</a>
<a href="https://github.com/axolotl-ai-cloud/axolotl/releases" style="text-decoration: none; margin: 0 10px;">
<img src="https://img.shields.io/github/release/axolotl-ai-cloud/axolotl.svg?style=for-the-badge&color=purple" alt="Releases">
</a>
</div>
<div align="center" style="margin: 30px 0;">
<a href="https://discord.com/invite/HhrNrHJPRb" style="text-decoration: none; margin: 0 10px;">
<img src="https://img.shields.io/badge/Discord-Join%20Community-7289da?style=for-the-badge&logo=discord&logoColor=white" alt="discord">
</a>
<a href="https://twitter.com/axolotl_ai" style="text-decoration: none; margin: 0 10px;">
<img src="https://img.shields.io/twitter/follow/axolotl_ai?style=for-the-badge&logo=twitter&color=1da1f2" alt="twitter">
</a>
</div>
---
<div style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); padding: 30px; border-radius: 15px; margin: 40px 0; color: white;">
<h2 style="color: white; text-align: center; margin-bottom: 20px;">
🎉 <strong>Latest Updates</strong>
</h2>
<div style="display: grid; gap: 15px;">
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/06:</strong> Magistral Support Added
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
Magistral with mistral-common tokenizer support has been added to Axolotl.
<a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/magistral" style="color: #fbbf24; text-decoration: none;">
<strong>See examples →</strong>
</a>
</p>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/05:</strong> Quantization Aware Training (QAT)
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
QAT support has been added to Axolotl.
<a href="https://docs.axolotl.ai/docs/qat.html" style="color: #fbbf24; text-decoration: none;">
<strong>Explore the docs →</strong>
</a>
</p>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/04:</strong> Llama 4 Support
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
Llama 4 support has been added in Axolotl.
<a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/llama-4" style="color: #fbbf24; text-decoration: none;">
<strong>See examples →</strong>
</a>
</p>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/03:</strong> Sequence Parallelism & Multimodal Support
</summary>
<div style="margin: 10px 0 0 0; padding-left: 20px;">
<p style="margin: 5px 0;">
• Sequence Parallelism (SP) for scaling context length -
<a href="https://huggingface.co/blog/axolotl-ai-co/long-context-with-sequence-parallelism-in-axolotl" style="color: #fbbf24; text-decoration: none;">
<strong>Blog</strong>
</a> |
<a href="https://docs.axolotl.ai/docs/sequence_parallelism.html" style="color: #fbbf24; text-decoration: none;">
<strong>Docs</strong>
</a>
</p>
<p style="margin: 5px 0;">
• (Beta) Multimodal models fine-tuning -
<a href="https://docs.axolotl.ai/docs/multimodal.html" style="color: #fbbf24; text-decoration: none;">
<strong>Check docs →</strong>
</a>
</p>
</div>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/02:</strong> LoRA Optimizations & GRPO Support
</summary>
<div style="margin: 10px 0 0 0; padding-left: 20px;">
<p style="margin: 5px 0;">
• LoRA optimizations for better memory usage and speed -
<a href="https://docs.axolotl.ai/docs/lora_optims.html" style="color: #fbbf24; text-decoration: none;">
<strong>Docs →</strong>
</a>
</p>
<p style="margin: 5px 0;">
• GRPO support added -
<a href="https://huggingface.co/blog/axolotl-ai-co/training-llms-w-interpreter-feedback-wasm" style="color: #fbbf24; text-decoration: none;">
<strong>Blog</strong>
</a> |
<a href="https://github.com/axolotl-ai-cloud/grpo_code" style="color: #fbbf24; text-decoration: none;">
<strong>Example</strong>
</a>
</p>
</div>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/01:</strong> Reward Modelling Support
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
Reward Modelling / Process Reward Modelling fine-tuning support added.
<a href="https://docs.axolotl.ai/docs/reward_modelling.html" style="color: #fbbf24; text-decoration: none;">
<strong>See docs →</strong>
</a>
</p>
</details>
</div>
</div>
---
<div style="background: linear-gradient(135deg, #ff6b6b 0%, #feca57 100%); padding: 30px; border-radius: 15px; margin: 40px 0; color: white;">
<h2 style="color: white; text-align: center; margin-bottom: 30px;">
✨ <strong>What Makes Axolotl Special</strong>
</h2>
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 20px;">
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">🚀 Multiple Model Support</h3>
<p style="margin: 0; line-height: 1.6;">
Train LLaMA, Mistral, Mixtral, Pythia, and more. Full compatibility with HuggingFace transformers causal language models.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">🎯 Advanced Training Methods</h3>
<p style="margin: 0; line-height: 1.6;">
Full fine-tuning, LoRA, QLoRA, GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO), Multimodal, and Reward Modelling.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">⚙️ Easy Configuration</h3>
<p style="margin: 0; line-height: 1.6;">
Reuse a single YAML file across dataset preprocessing, training, evaluation, quantization, and inference.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">⚡ Performance Optimizations</h3>
<p style="margin: 0; line-height: 1.6;">
Multipacking, Flash Attention, Xformers, Flex Attention, Liger Kernel, Sequence Parallelism, and Multi-GPU training.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">📊 Flexible Dataset Handling</h3>
<p style="margin: 0; line-height: 1.6;">
Load from local files, HuggingFace datasets, and cloud storage (S3, Azure, GCP, OCI).
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">☁️ Cloud Ready</h3>
<p style="margin: 0; line-height: 1.6;">
Pre-built Docker images and PyPI packages for seamless deployment on cloud platforms and local hardware.
</p>
</div>
</div>
</div>
---
<div style="background: linear-gradient(135deg, #4facfe 0%, #00f2fe 100%); padding: 30px; border-radius: 15px; margin: 40px 0; color: white;">
<h2 style="color: white; text-align: center; margin-bottom: 20px;">
🚀 <strong>Quick Start</strong>
</h2>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; margin: 20px 0;">
<h3 style="color: #fbbf24; margin-bottom: 15px;">📋 Requirements</h3>
<ul style="margin: 0; padding-left: 20px; line-height: 1.8;">
<li><strong>GPU:</strong> NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU</li>
<li><strong>Python:</strong> 3.11+</li>
<li><strong>PyTorch:</strong> ≥2.5.1</li>
</ul>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; margin: 20px 0;">
<h3 style="color: #fbbf24; margin-bottom: 15px;">💾 Installation</h3>
<div style="background: rgba(0,0,0,0.3); padding: 15px; border-radius: 8px; font-family: 'Courier New', monospace; overflow-x: auto;">
<pre style="margin: 0; color: #fff;"><code># Install dependencies
pip3 install -U packaging==23.2 setuptools==75.8.0 wheel ninja
# Install Axolotl with Flash Attention and DeepSpeed
pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]
# Download examples and configs
axolotl fetch examples
axolotl fetch deepspeed_configs # OPTIONAL</code></pre>
</div>
<p style="margin: 15px 0 0 0; font-size: 14px; opacity: 0.9;">
<em>Other installation methods available in our <a href="https://docs.axolotl.ai/docs/installation.html" style="color: #fbbf24; text-decoration: none;"><strong>documentation</strong></a></em>
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; margin: 20px 0;">
<h3 style="color: #fbbf24; margin-bottom: 15px;">🎯 Your First Fine-tune</h3>
<div style="background: rgba(0,0,0,0.3); padding: 15px; border-radius: 8px; font-family: 'Courier New', monospace; overflow-x: auto;">
<pre style="margin: 0; color: #fff;"><code># Fetch examples
axolotl fetch examples
# Or specify custom path
axolotl fetch examples --dest path/to/folder
# Start training with LoRA
axolotl train examples/llama-3/lora-1b.yml</code></pre>
</div>
<p style="margin: 15px 0 0 0; font-size: 14px; opacity: 0.9;">
<em>That's it! Check our <a href="https://docs.axolotl.ai/docs/getting-started.html" style="color: #fbbf24; text-decoration: none;"><strong>Getting Started Guide</strong></a> for detailed walkthrough</em>
</p>
</div>
</div>
---
<div style="background: linear-gradient(135deg, #a8edea 0%, #fed6e3 100%); padding: 30px; border-radius: 15px; margin: 40px 0; color: #374151;">
<h2 style="color: #374151; text-align: center; margin-bottom: 30px;">
📚 <strong>Documentation Hub</strong>
</h2>
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); gap: 15px;">
<a href="https://docs.axolotl.ai/docs/installation.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1); transition: transform 0.2s; hover: transform: translateY(-2px);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🔧 Installation Options</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Detailed setup instructions for different environments</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/config.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">⚙️ Configuration Guide</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Full configuration options and examples</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/dataset_loading.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">📊 Dataset Loading</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Loading datasets from various sources</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/dataset-formats/" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">📋 Dataset Guide</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Supported formats and usage instructions</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/multi-gpu.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🖥️ Multi-GPU Training</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Scale your training across multiple GPUs</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/multi-node.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🌐 Multi-Node Training</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Distributed training across multiple machines</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/multipack.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">📦 Multipacking</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Efficient batch packing for training</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/api/" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🔍 API Reference</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Auto-generated code documentation</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/faq.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">❓ FAQ</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Frequently asked questions</p>
</div>
</a>
</div>
</div>
---
<div style="background: linear-gradient(135deg, #ffecd2 0%, #fcb69f 100%); padding: 30px; border-radius: 15px; margin: 40px 0; color: #78350f;">
<h2 style="color: #78350f; text-align: center; margin-bottom: 20px;">
🤝 <strong>Getting Help</strong>
</h2>
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 20px;">
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">💬</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Community Support</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Join thousands of developers in our Discord</p>
<a href="https://discord.gg/HhrNrHJPRb" style="background: #7289da; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">Join Discord</a>
</div>
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">📖</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Examples</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Browse our comprehensive examples</p>
<a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/" style="background: #10b981; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">View Examples</a>
</div>
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">🔧</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Debugging</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Troubleshooting and debugging guide</p>
<a href="https://docs.axolotl.ai/docs/debugging.html" style="background: #f59e0b; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">Debug Guide</a>
</div>
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">✉️</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Enterprise Support</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Need dedicated support? Contact us</p>
<a href="mailto:wing@axolotl.ai" style="background: #8b5cf6; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">Contact Us</a>
</div>
</div>
</div>
---
<div style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); padding: 30px; border-radius: 15px; margin: 40px 0; color: white;">
<h2 style="color: white; text-align: center; margin-bottom: 20px;">
🌟 <strong>Contributing</strong>
</h2>
<div style="text-align: center; margin-bottom: 20px;">
<p style="font-size: 16px; margin-bottom: 20px;">
We welcome contributions from the community! Whether it's bug fixes,