tocmo0nlord/axolotl

Files

Dan Saunders 69df309cbb separate out flash-attn install (sadly)

2025-09-30 14:58:56 -04:00

2.0 KiB

Raw Blame History

Finetune ByteDance's Seed-OSS with Axolotl

Seed-OSS are a series of 36B parameter open source models trained by ByteDance's Seed Team.

This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.

Getting started

Install Axolotl following the installation guide. You need to install from main as Seed-OSS is only on nightly or use our latest Docker images.

Here is an example of how to install from main for pip:

# Ensure you have Pytorch installed (Pytorch 2.6.0 min)
git clone https://github.com/axolotl-ai-cloud/axolotl.git
cd axolotl

uv sync --extra deepspeed
uv pip install flash-attn --no-build-isolation

# Install Cut Cross Entropy
python scripts/cutcrossentropy_install.py | sh

Run the finetuning example:

axolotl train examples/seed-oss/seed-oss-36b-qlora.yaml

This config uses about 27.7 GiB VRAM.

Let us know how it goes. Happy finetuning! 🚀

TIPS

For inference, the official Seed Team recommends top_p=0.95 and temperature=1.1.
You can run a full finetuning by removing the adapter: qlora and load_in_4bit: true from the config.
Read more on how to load your own dataset at docs.
The dataset format follows the OpenAI Messages format as seen here.

Optimization Guides