# Ministral

Ministral is a family of openweight models from MistralAI found on HuggingFace. This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.

## Getting started

1. Install Axolotl following the installation guide.

2. Install Cut Cross Entropy to reduce training VRAM usage.

3. Run the finetuning example:

       axolotl train examples/ministral/ministral-small-qlora.yaml

This config uses about 8.76 GiB VRAM.

Let us know how it goes. Happy finetuning! 🚀

### Tips

- We recommend adding the same/similar SystemPrompt that the model is tuned for. You can find this within the repo’s files titled `SYSTEM_PROMPT.txt`.
- You can run a full finetuning by removing the `adapter: qlora` and `load_in_4bit: true` from the config.
- Read more on how to load your own dataset at docs.
- The text dataset format follows the OpenAI Messages format as seen here.

## Optimization Guides

Please check the Optimizations doc.

## Limitations

We only support the `mistral-common` tokenizer for Supervised Fine-tuning at the moment and for `type: chat_template` only.

In addition, we do not support overriding tokens yet.

## Related Resources

- MistralAI Ministral Blog
- Axolotl Docs
- Axolotl Website
- Axolotl GitHub
- Axolotl Discord

## Future Work

- Add parity to Preference Tuning, RL, etc.
- Add parity to other tokenizer configs like overriding tokens.