# Ministral3

Ministral3 is a family of open-weight models from MistralAI found on HuggingFace. This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.

Please see Thinking and Vision for their respective fine-tuning.

Thanks to the team at MistralAI for giving us early access to prepare for these releases.

Note: This is still experimental given it is based on transformers v5 RC.

## Getting started

1. Install Axolotl from source following the installation guide.

2. Install Cut Cross Entropy to reduce training VRAM usage.

3. Swap to the Axolotl transformers v5 branch

       cp examples/ministral3/ministral3-3b-qlora.yaml ministral3-3b-qlora.yaml

       git fetch
       git checkout transformers-v5

       # Install packages for transformers v5
       pip install -e .

4. Run the fine-tuning:

       axolotl train ministral3-3b-qlora.yaml

Let us know how it goes. Happy finetuning! 🚀

### Tips

- We recommend adding the same/similar SystemPrompt that the model is tuned for. You can find this within the repo’s files titled `SYSTEM_PROMPT.txt`.
- You can run a full finetuning by removing the `adapter: qlora` and `load_in_4bit: true` from the config.
- Read more on how to load your own dataset at docs.
- The text dataset format follows the OpenAI Messages format as seen here.

### Thinking

Ministral3 2512 model supports thinking capabilities, enabling Chain-of-Thought reasoning with explicit thinking steps.

📚 See the Thinking fine-tuning guide →

### Vision

Ministral3 2512 model also supports vision capabilities.

📚 See the Vision fine-tuning guide →

## Optimization Guides

Please check the Optimizations doc.

## Limitations

We only support the `mistral-common` tokenizer for Supervised Fine-tuning at the moment and for `type: chat_template` only.

In addition, we do not support overriding tokens yet.

## Related Resources

- MistralAI Mistral3 Blog
- Axolotl Docs
- Axolotl Website
- Axolotl GitHub
- Axolotl Discord

## Future Work

- Add parity to Preference Tuning, RL, etc.
- Add parity to other tokenizer configs like overriding tokens.