2b66ee189c
* feat: add ministral and mistral3 * chore: lint * feat: update cce for ministral * fix: add vram usage * feat: update for release * fix: save_pretrained issue in v5 * fix: add instructions to use v5 branch * fix: add to multipack * fix: improve instructions * fix: add model to readme
Finetune Ministral with Axolotl
Ministral is a family of openweight models from MistralAI found on HuggingFace at 2410 and 2512 (see Thinking). This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.
Getting started
-
Install Axolotl following the installation guide.
-
Install Cut Cross Entropy to reduce training VRAM usage.
-
Run the finetuning example:
axolotl train examples/ministral/ministral-small-qlora.yaml
This config uses about 8.76 GiB VRAM.
Let us know how it goes. Happy finetuning! 🚀
Thinking
MistralAI has released their Ministral3 2512 model with thinking capabilities, enabling Chain-of-Thought reasoning with explicit thinking steps.
📚 See the Thinking fine-tuning guide →
For Ministral3 Base/Instruct, you can reuse the above config to train supervised finetuning.
Tips
- We recommend adding the same/similar SystemPrompt that the model is tuned for. You can find this within the repo's files titled
SYSTEM_PROMPT.txt. - You can run a full finetuning by removing the
adapter: qloraandload_in_4bit: truefrom the config. - Read more on how to load your own dataset at docs.
- The text dataset format follows the OpenAI Messages format as seen here.
Optimization Guides
Please check the Optimizations doc.
Limitations
We only support the mistral-common tokenizer for Supervised Fine-tuning at the moment and for type: chat_template only.
In addition, we do not support overriding tokens yet.
Related Resources
Future Work
- Add parity to Preference Tuning, RL, etc.
- Add parity to other tokenizer configs like overriding tokens.