Files

NanoCode012 9de5b76336 feat: move to uv first (#3545 )

* feat: move to uv first

* fix: update doc to uv first

* fix: merge dev/tests into uv pyproject

* fix: update docker docs to match current config

* fix: migrate examples to readme

* fix: add llmcompressor to conflict

* feat: rec uv sync with lockfile for dev/ci

* fix: update docker docs to clarify how to use uv images

* chore: docs

* fix: use system python, no venv

* fix: set backend cpu

* fix: only set for installing pytorch step

* fix: remove unsloth kernel and installs

* fix: remove U in tests

* fix: set backend in deps too

* chore: test

* chore: comments

* fix: attempt to lock torch

* fix: workaround torch cuda and not upgraded

* fix: forgot to push

* fix: missed source

* fix: nightly upstream loralinear config

* fix: nightly phi3 long rope not work

* fix: forgot commit

* fix: test phi3 template change

* fix: no more requirements

* fix: carry over changes from new requirements to pyproject

* chore: remove lockfile per discussion

* fix: set match-runtime

* fix: remove unneeded hf hub buildtime

* fix: duplicate cache delete on nightly

* fix: torchvision being overridden

* fix: migrate to uv images

* fix: leftover from merge

* fix: simplify base readme

* fix: update assertion message to be clearer

* chore: docs

* fix: change fallback for cicd script

* fix: match against main exactly

* fix: peft 0.19.1 change

* fix: e2e test

* fix: ci

* fix: e2e test

2026-04-21 10:16:03 -04:00

2.7 KiB

Raw Blame History

Finetune Ministral3 with Axolotl

Ministral3 is a family of open-weight models from MistralAI found on HuggingFace. This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.

Please see Thinking and Vision for their respective fine-tuning.

Thanks to the team at MistralAI for giving us early access to prepare for these releases.

Note: This is still experimental given it is based on transformers v5 RC.

Getting started

Install Axolotl from source following the installation guide.
Install Cut Cross Entropy to reduce training VRAM usage.

Swap to the Axolotl transformers v5 branch

cp examples/ministral3/ministral3-3b-qlora.yaml ministral3-3b-qlora.yaml

git fetch
git checkout transformers-v5

# Install packages for transformers v5
uv pip install -e .

Run the fine-tuning:
```
axolotl train ministral3-3b-qlora.yaml
```

Let us know how it goes. Happy finetuning! 🚀

Tips

We recommend adding the same/similar SystemPrompt that the model is tuned for. You can find this within the repo's files titled SYSTEM_PROMPT.txt.
You can run a full finetuning by removing the adapter: qlora and load_in_4bit: true from the config.
Read more on how to load your own dataset at docs.
The text dataset format follows the OpenAI Messages format as seen here.

Thinking

Ministral3 2512 model supports thinking capabilities, enabling Chain-of-Thought reasoning with explicit thinking steps.

📚 See the Thinking fine-tuning guide →

Vision

Ministral3 2512 model also supports vision capabilities.

📚 See the Vision fine-tuning guide →

Optimization Guides

Please check the Optimizations doc.

Limitations

We only support the mistral-common tokenizer for Supervised Fine-tuning at the moment and for type: chat_template only.

In addition, we do not support overriding tokens yet.

Future Work

Add parity to Preference Tuning, RL, etc.
Add parity to other tokenizer configs like overriding tokens.

2.7 KiB Raw Blame History