79103b01ca
* feat: add seedoss cce * feat: add seedoss config and docs * fix: shouldn't have target modules with target linear * feat: add vram numbers * fix: hf link * fix: name * fix: support multipack seedoss * fix: merge error * feat: update seedoss instructions for transformers release
2.1 KiB
2.1 KiB
Finetune ByteDance's Seed-OSS with Axolotl
Seed-OSS are a series of 36B parameter open source models trained by ByteDance's Seed Team.
This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.
Getting started
-
Install Axolotl following the installation guide. You need to install from main as Seed-OSS is only on nightly or use our latest Docker images.
Here is an example of how to install from main for pip:
# Ensure you have Pytorch installed (Pytorch 2.6.0 min)
git clone https://github.com/axolotl-ai-cloud/axolotl.git
cd axolotl
pip3 install packaging==23.2 setuptools==75.8.0 wheel ninja
pip3 install --no-build-isolation -e '.[flash-attn]'
# Install Cut Cross Entropy
python scripts/cutcrossentropy_install.py | sh
- Run the finetuning example:
axolotl train examples/seed-oss/seed-oss-36b-qlora.yaml
This config uses about 27.7 GiB VRAM.
Let us know how it goes. Happy finetuning! 🚀
TIPS
- For inference, the official Seed Team recommends
top_p=0.95andtemperature=1.1. - You can run a full finetuning by removing the
adapter: qloraandload_in_4bit: truefrom the config. - Read more on how to load your own dataset at docs.
- The dataset format follows the OpenAI Messages format as seen here.