🧠 Streamlined AI Model Post-Training
A powerful, flexible tool designed to streamline post-training for various AI models with enterprise-grade features and optimizations.
|
|
|
|
🎉 Latest Updates
📅 2025/06: Magistral Support Added
Magistral with mistral-common tokenizer support has been added to Axolotl. See examples →
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/05:</strong> Quantization Aware Training (QAT)
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
QAT support has been added to Axolotl.
<a href="https://docs.axolotl.ai/docs/qat.html" style="color: #fbbf24; text-decoration: none;">
<strong>Explore the docs →</strong>
</a>
</p>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/04:</strong> Llama 4 Support
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
Llama 4 support has been added in Axolotl.
<a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/llama-4" style="color: #fbbf24; text-decoration: none;">
<strong>See examples →</strong>
</a>
</p>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/03:</strong> Sequence Parallelism & Multimodal Support
</summary>
<div style="margin: 10px 0 0 0; padding-left: 20px;">
<p style="margin: 5px 0;">
• Sequence Parallelism (SP) for scaling context length -
<a href="https://huggingface.co/blog/axolotl-ai-co/long-context-with-sequence-parallelism-in-axolotl" style="color: #fbbf24; text-decoration: none;">
<strong>Blog</strong>
</a> |
<a href="https://docs.axolotl.ai/docs/sequence_parallelism.html" style="color: #fbbf24; text-decoration: none;">
<strong>Docs</strong>
</a>
</p>
<p style="margin: 5px 0;">
• (Beta) Multimodal models fine-tuning -
<a href="https://docs.axolotl.ai/docs/multimodal.html" style="color: #fbbf24; text-decoration: none;">
<strong>Check docs →</strong>
</a>
</p>
</div>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/02:</strong> LoRA Optimizations & GRPO Support
</summary>
<div style="margin: 10px 0 0 0; padding-left: 20px;">
<p style="margin: 5px 0;">
• LoRA optimizations for better memory usage and speed -
<a href="https://docs.axolotl.ai/docs/lora_optims.html" style="color: #fbbf24; text-decoration: none;">
<strong>Docs →</strong>
</a>
</p>
<p style="margin: 5px 0;">
• GRPO support added -
<a href="https://huggingface.co/blog/axolotl-ai-co/training-llms-w-interpreter-feedback-wasm" style="color: #fbbf24; text-decoration: none;">
<strong>Blog</strong>
</a> |
<a href="https://github.com/axolotl-ai-cloud/grpo_code" style="color: #fbbf24; text-decoration: none;">
<strong>Example</strong>
</a>
</p>
</div>
</details>
<details style="background: rgba(255,255,255,0.1); padding: 15px; border-radius: 10px; border: 1px solid rgba(255,255,255,0.2);">
<summary style="font-weight: 600; cursor: pointer; padding: 10px 0;">
<strong>📅 2025/01:</strong> Reward Modelling Support
</summary>
<p style="margin: 10px 0 0 0; padding-left: 20px;">
Reward Modelling / Process Reward Modelling fine-tuning support added.
<a href="https://docs.axolotl.ai/docs/reward_modelling.html" style="color: #fbbf24; text-decoration: none;">
<strong>See docs →</strong>
</a>
</p>
</details>
✨ What Makes Axolotl Special
🚀 Multiple Model Support
Train LLaMA, Mistral, Mixtral, Pythia, and more. Full compatibility with HuggingFace transformers causal language models.
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">🎯 Advanced Training Methods</h3>
<p style="margin: 0; line-height: 1.6;">
Full fine-tuning, LoRA, QLoRA, GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO), Multimodal, and Reward Modelling.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">⚙️ Easy Configuration</h3>
<p style="margin: 0; line-height: 1.6;">
Reuse a single YAML file across dataset preprocessing, training, evaluation, quantization, and inference.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">⚡ Performance Optimizations</h3>
<p style="margin: 0; line-height: 1.6;">
Multipacking, Flash Attention, Xformers, Flex Attention, Liger Kernel, Sequence Parallelism, and Multi-GPU training.
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">📊 Flexible Dataset Handling</h3>
<p style="margin: 0; line-height: 1.6;">
Load from local files, HuggingFace datasets, and cloud storage (S3, Azure, GCP, OCI).
</p>
</div>
<div style="background: rgba(255,255,255,0.1); padding: 20px; border-radius: 12px; border: 1px solid rgba(255,255,255,0.2);">
<h3 style="color: #fbbf24; margin-bottom: 10px;">☁️ Cloud Ready</h3>
<p style="margin: 0; line-height: 1.6;">
Pre-built Docker images and PyPI packages for seamless deployment on cloud platforms and local hardware.
</p>
</div>
🚀 Quick Start
📋 Requirements
- GPU: NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU
- Python: 3.11+
- PyTorch: ≥2.5.1
💾 Installation
# Install dependencies pip3 install -U packaging==23.2 setuptools==75.8.0 wheel ninjaInstall Axolotl with Flash Attention and DeepSpeed
pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]
Download examples and configs
axolotl fetch examples axolotl fetch deepspeed_configs # OPTIONAL
Other installation methods available in our documentation
🎯 Your First Fine-tune
# Fetch examples axolotl fetch examplesOr specify custom path
axolotl fetch examples --dest path/to/folder
Start training with LoRA
axolotl train examples/llama-3/lora-1b.yml
That's it! Check our Getting Started Guide for detailed walkthrough
📚 Documentation Hub
🔧 Installation Options
Detailed setup instructions for different environments
<a href="https://docs.axolotl.ai/docs/config.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">⚙️ Configuration Guide</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Full configuration options and examples</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/dataset_loading.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">📊 Dataset Loading</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Loading datasets from various sources</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/dataset-formats/" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">📋 Dataset Guide</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Supported formats and usage instructions</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/multi-gpu.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🖥️ Multi-GPU Training</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Scale your training across multiple GPUs</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/multi-node.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🌐 Multi-Node Training</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Distributed training across multiple machines</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/multipack.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">📦 Multipacking</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Efficient batch packing for training</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/api/" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">🔍 API Reference</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Auto-generated code documentation</p>
</div>
</a>
<a href="https://docs.axolotl.ai/docs/faq.html" style="text-decoration: none; color: inherit;">
<div style="background: rgba(255,255,255,0.7); padding: 15px; border-radius: 10px; border: 1px solid rgba(55,65,81,0.1);">
<h4 style="margin: 0 0 8px 0; color: #1f2937;">❓ FAQ</h4>
<p style="margin: 0; font-size: 14px; opacity: 0.8;">Frequently asked questions</p>
</div>
</a>
🤝 Getting Help
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">📖</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Examples</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Browse our comprehensive examples</p>
<a href="https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/" style="background: #10b981; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">View Examples</a>
</div>
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">🔧</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Debugging</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Troubleshooting and debugging guide</p>
<a href="https://docs.axolotl.ai/docs/debugging.html" style="background: #f59e0b; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">Debug Guide</a>
</div>
<div style="background: rgba(255,255,255,0.6); padding: 20px; border-radius: 12px; text-align: center;">
<div style="font-size: 30px; margin-bottom: 10px;">✉️</div>
<h3 style="margin: 0 0 10px 0; color: #78350f;">Enterprise Support</h3>
<p style="margin: 0 0 15px 0; font-size: 14px;">Need dedicated support? Contact us</p>
<a href="mailto:wing@axolotl.ai" style="background: #8b5cf6; color: white; padding: 8px 16px; border-radius: 6px; text-decoration: none; font-weight: 600;">Contact Us</a>
</div>
🌟 Contributing
We welcome contributions from the community! Whether it's bug fixes,