axolotl/README.md

Axolotl: Fine-tune LLMs with Unprecedented Ease & Power! 🚀

Your ultimate toolkit for efficient, scalable, and versatile large language model fine-tuning.

---

🎉 Latest Innovations & Updates!

• 2025/06: Magistral with mistral-common tokenizer support! Dive into examples to train your own Magistral models.
• 2025/05: Quantization Aware Training (QAT) support! Explore the docs to learn more.
• 2025/04: Llama 4 support! See examples to train Llama 4 with Axolotl's linearized version!
• 2025/03: Sequence Parallelism (SP) support! Scale your context length. Read the blog and docs.
• 2025/03: (Beta) Fine-tuning Multimodal models! Check out the docs.
• 2025/02: LoRA optimizations! Reduce memory and improve speed. Jump into the docs.
• 2025/02: GRPO support! Dive into our blog and GRPO example.
• 2025/01: Reward Modelling / Process Reward Modelling fine-tuning! See docs.

✨ Axolotl Overview: Your LLM Fine-tuning Powerhouse!

Axolotl is a powerful, flexible, and user-friendly tool designed to supercharge your post-training workflows for a wide range of cutting-edge AI models.

🤖 Broad Model Compatibility

• Train a vast array of models including LLaMA, Mistral, Mixtral, Pythia, and many more.
• Fully compatible with HuggingFace transformers causal language models, ensuring wide adoption.

🔧 Diverse Training Methodologies

• Full fine-tuning, LoRA, QLoRA, GPTQ, QAT.
• Preference Tuning: DPO, IPO, KTO, ORPO.
• Advanced RL: GRPO.
• Multimodal and Reward Modelling (RM) / Process Reward Modelling (PRM).

⚙️ Streamlined Configuration

• Utilize a single, intuitive YAML file across dataset preprocess, training, evaluation, quantization, and inference.

⚡ Cutting-Edge Performance Optimizations

• Multipacking, Flash Attention, Xformers, Flex Attention, Liger Kernel, Cut Cross Entropy.
• Sequence Parallelism (SP), LoRA optimizations.
• Multi-GPU training (FSDP1, FSDP2, DeepSpeed), Multi-node training (Torchrun, Ray), and many more!

📂 Flexible Data Handling

• Load datasets from local paths, HuggingFace Hub, and major cloud providers (S3, Azure, GCP, OCI).

☁️ Cloud-Ready & Deployable

• Official Docker images and PyPI packages for seamless integration on cloud platforms and local hardware.

🚀 Quick Start: Get Fine-tuning in Minutes!

Requirements:

• ▶ NVIDIA GPU (Ampere or newer for `bf16` and Flash Attention) or AMD GPU
• ▶ Python 3.11
• ▶ PyTorch ≥2.5.1

Installation:

pip3 install -U packaging==23.2 setuptools==75.8.0 wheel ninja
pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]
Download example axolotl configs, deepspeed configs
axolotl fetch examples
axolotl fetch deepspeed_configs  # OPTIONAL

Other installation approaches are described here.

Your First Fine-tune:

# Fetch axolotl examples
axolotl fetch examples
Or, specify a custom path
axolotl fetch examples --dest path/to/folder
Train a model using LoRA
axolotl train examples/llama-3/lora-1b.yml

That's it! Check out our  Getting Started Guide ➜ for a more detailed walkthrough.

📚 Comprehensive Documentation: Unlock Axolotl's Full Potential

Dive deep into Axolotl's capabilities with our extensive documentation:

•  Installation Options - Detailed setup instructions for different environments
•  Configuration Guide - Full configuration options and examples
•  Dataset Loading - Loading datasets from various sources
•  Dataset Guide - Supported formats and how to use them
•  Multi-GPU Training
•  Multi-Node Training
•  Multipacking
•  API Reference - Auto-generated code documentation
• ❓ FAQ - Frequently asked questions

🤝 Need Help? We're Here for You!

•  Join our vibrant Discord community for real-time support and discussions.
•  Explore our Examples directory for practical use cases.
•  Read our Debugging Guide for troubleshooting tips.
• ✉ Need dedicated support? Please contact wing@axolotl.ai for professional assistance options.

🌟 Contribute to Axolotl!

Contributions are always welcome and highly appreciated! Axolotl thrives on community support. Please see our Contributing Guide for details on how you can help make Axolotl even better.

❤️ Our Esteemed Sponsors

A huge thank you to our visionary sponsors who provide the essential resources to keep Axolotl at the forefront of LLM fine-tuning:

Modal: Revolutionizing cloud computing for Gen AI. Run jobs, deploy models, and fine-tune LLMs at scale with ease.

Interested in powering the future of Axolotl? Become a sponsor! Contact us at wing@axolotl.ai

📜 License

This project is proudly licensed under the Apache 2.0 License. See the LICENSE file for full details.