# Axolotl

## 🎉 Latest Updates

- 2025/07: Voxtral with mistral-common tokenizer support has been integrated in Axolotl. Read the docs!
- 2025/07: TiledMLP support for single-GPU to multi-GPU training with DDP, DeepSpeed and FSDP support has been added to support Arctic Long Sequence Training. (ALST). See examples for using ALST with Axolotl!
- 2025/06: Magistral with mistral-common tokenizer support has been added to Axolotl. See examples to start training your own Magistral models with Axolotl!
- 2025/05: Quantization Aware Training (QAT) support has been added to Axolotl. Explore the docs to learn more!
- 2025/04: Llama 4 support has been added in Axolotl. See examples to start training your own Llama 4 models with Axolotl’s linearized version!
- 2025/03: Axolotl has implemented Sequence Parallelism (SP) support. Read the blog and docs to learn how to scale your context length when fine-tuning.
- 2025/03: (Beta) Fine-tuning Multimodal models is now supported in Axolotl. Check out the docs to fine-tune your own!
- 2025/02: Axolotl has added LoRA optimizations to reduce memory usage and improve training speed for LoRA and QLoRA in single GPU and multi-GPU training (DDP and DeepSpeed). Jump into the docs to give it a try.
- 2025/02: Axolotl has added GRPO support. Dive into our blog and GRPO example and have some fun!
- 2025/01: Axolotl has added Reward Modelling / Process Reward Modelling fine-tuning support. See docs.

## ✨ Overview

Axolotl is a tool designed to streamline post-training for various AI models.

Features:

- Multiple Model Support: Train various models like LLaMA, Mistral, Mixtral, Pythia, and more. We are compatible with HuggingFace transformers causal language models.
- Training Methods: Full fine-tuning, LoRA, QLoRA, GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO), Multimodal, and Reward Modelling (RM) / Process Reward Modelling (PRM).
- Easy Configuration: Re-use a single YAML file between dataset preprocess, training, evaluation, quantization, and inference.
- Performance Optimizations: Multipacking, Flash Attention, Xformers, Flex Attention, Liger Kernel, Cut Cross Entropy, Sequence Parallelism (SP), LoRA optimizations, Multi-GPU training (FSDP1, FSDP2, DeepSpeed), Multi-node training (Torchrun, Ray), and many more!
- Flexible Dataset Handling: Load from local, HuggingFace, and cloud (S3, Azure, GCP, OCI) datasets.
- Cloud Ready: We ship Docker images and also PyPI packages for use on cloud platforms and local hardware.

## 🚀 Quick Start

Requirements:

- NVIDIA GPU (Ampere or newer for `bf16` and Flash Attention) or AMD GPU
- Python 3.11
- PyTorch ≥2.6.0

### Installation

#### Using pip

    pip3 install -U packaging==23.2 setuptools==75.8.0 wheel ninja
    pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]

    # Download example axolotl configs, deepspeed configs
    axolotl fetch examples
    axolotl fetch deepspeed_configs  # OPTIONAL

#### Using Docker

Installing with Docker can be less error prone than installing in your own environment.

    docker run --gpus '"all"' --rm -it axolotlai/axolotl:main-latest

Other installation approaches are described here.

#### Cloud Providers

- RunPod
- Vast.ai
- PRIME Intellect
- Modal
- Novita
- JarvisLabs.ai
- Latitude.sh

### Your First Fine-tune

    # Fetch axolotl examples
    axolotl fetch examples

    # Or, specify a custom path
    axolotl fetch examples --dest path/to/folder

    # Train a model using LoRA
    axolotl train examples/llama-3/lora-1b.yml

That’s it! Check out our Getting Started Guide for a more detailed walkthrough.

## 📚 Documentation

- Installation Options - Detailed setup instructions for different environments
- Configuration Guide - Full configuration options and examples
- Dataset Loading - Loading datasets from various sources
- Dataset Guide - Supported formats and how to use them
- Multi-GPU Training
- Multi-Node Training
- Multipacking
- API Reference - Auto-generated code documentation
- FAQ - Frequently asked questions

## 🤝 Getting Help

- Join our Discord community for support
- Check out our Examples directory
- Read our Debugging Guide
- Need dedicated support? Please contact ✉️wing@axolotl.ai for options

## 🌟 Contributing

Contributions are welcome! Please see our Contributing Guide for details.

## ❤️ Sponsors

Interested in sponsoring? Contact us at wing@axolotl.ai

## 📜 License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.