diff --git a/README.md b/README.md index 1517fb874..13518f2a8 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ ## 🎉 Latest Updates -- 2025/11: Axolotl now includes support for [Olmo3](https://github.com/axolotl-ai-cloud/axolotl/blob/main/examples/olmo3). +- 2025/12: Axolotl now includes support for [Olmo3](https://github.com/axolotl-ai-cloud/axolotl/blob/main/examples/olmo3), [Trinity](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/trinity), and [Ministral3](https://github.com/axolotl-ai-cloud/axolotl/blob/main/examples/ministral). - 2025/10: New model support has been added in Axolotl for: [Qwen3 Next](https://github.com/axolotl-ai-cloud/axolotl/blob/main/examples/qwen3-next), [Qwen2.5-vl, Qwen3-vl](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/qwen2_5-vl), [Qwen3, Qwen3MoE](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/qwen3), [Granite 4](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/granite4), [HunYuan](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/hunyuan), [Magistral 2509](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/magistral#vision), [Apertus](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/apertus), and [Seed-OSS](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/seed-oss). - 2025/09: Axolotl now has text diffusion training. Read more [here](https://github.com/axolotl-ai-cloud/axolotl/tree/main/src/axolotl/integrations/diffusion). - 2025/08: QAT has been updated to include NVFP4 support. See [PR](https://github.com/axolotl-ai-cloud/axolotl/pull/3107). diff --git a/examples/colab-notebooks/colab-axolotl-example.ipynb b/examples/colab-notebooks/colab-axolotl-example.ipynb index 57a638948..06705eb3d 100644 --- a/examples/colab-notebooks/colab-axolotl-example.ipynb +++ b/examples/colab-notebooks/colab-axolotl-example.ipynb @@ -40,7 +40,7 @@ "%%capture\n", "# This step can take ~5-10 minutes to install dependencies\n", "!pip install --no-build-isolation axolotl[flash-attn]>=0.9.1\n", - "!pip install \"cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@5eff953\"" + "!pip install \"cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@f643b88\"" ] }, { diff --git a/examples/magistral/README.md b/examples/magistral/README.md index a09138744..40a793f10 100644 --- a/examples/magistral/README.md +++ b/examples/magistral/README.md @@ -13,7 +13,7 @@ Thanks to the team at MistralAI for giving us early access to prepare for these Here is an example of how to install from pip: ```bash -# Ensure you have Pytorch installed (Pytorch 2.6.0 min) +# Ensure you have Pytorch installed (Pytorch 2.7.0 min) pip3 install packaging==23.2 setuptools==75.8.0 wheel ninja pip3 install --no-build-isolation 'axolotl[flash-attn]>=0.12.0' ``` diff --git a/examples/ministral/README.md b/examples/ministral/README.md new file mode 100644 index 000000000..b088c06ec --- /dev/null +++ b/examples/ministral/README.md @@ -0,0 +1,58 @@ +# Finetune Ministral with Axolotl + +Ministral is a family of openweight models from MistralAI found on HuggingFace at [2410](mistralai/Ministral-8B-Instruct-2410) and [2512](https://huggingface.co/collections/mistralai/ministral-3) (see [Thinking](#thinking)). This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking. + +## Getting started + +1. Install Axolotl following the [installation guide](https://docs.axolotl.ai/docs/installation.html). + +2. Install [Cut Cross Entropy](https://docs.axolotl.ai/docs/custom_integrations.html#cut-cross-entropy) to reduce training VRAM usage. + +3. Run the finetuning example: + + ```bash + axolotl train examples/ministral/ministral-small-qlora.yaml + ``` + +This config uses about 8.76 GiB VRAM. + +Let us know how it goes. Happy finetuning! 🚀 + +### Thinking + +MistralAI has released their [Ministral3 2512](https://huggingface.co/collections/mistralai/ministral-3) model with thinking capabilities, enabling Chain-of-Thought reasoning with explicit thinking steps. + +📚 **[See the Thinking fine-tuning guide →](./think/README.md)** + +For Ministral3 Base/Instruct, you can reuse the above config to train supervised finetuning. + +### Tips + +- We recommend adding the same/similar SystemPrompt that the model is tuned for. You can find this within the repo's files titled `SYSTEM_PROMPT.txt`. +- You can run a full finetuning by removing the `adapter: qlora` and `load_in_4bit: true` from the config. +- Read more on how to load your own dataset at [docs](https://docs.axolotl.ai/docs/dataset_loading.html). +- The text dataset format follows the OpenAI Messages format as seen [here](https://docs.axolotl.ai/docs/dataset-formats/conversation.html#chat_template). + +## Optimization Guides + +Please check the [Optimizations doc](https://docs.axolotl.ai/docs/optimizations.html). + +## Limitations + +We only support the `mistral-common` tokenizer for Supervised Fine-tuning at the moment and for `type: chat_template` only. + +In addition, we do not support overriding tokens yet. + +## Related Resources + +- [MistralAI Ministral Blog](https://mistral.ai/news/ministraux) +- [Axolotl Docs](https://docs.axolotl.ai) +- [Axolotl Website](https://axolotl.ai) +- [Axolotl GitHub](https://github.com/axolotl-ai-cloud/axolotl) +- [Axolotl Discord](https://discord.gg/7m9sfhzaf3) + + +## Future Work + +- Add parity to Preference Tuning, RL, etc. +- Add parity to other tokenizer configs like overriding tokens. diff --git a/examples/ministral/ministral-small-qlora.yaml b/examples/ministral/ministral-small-qlora.yaml new file mode 100644 index 000000000..0d5300ef6 --- /dev/null +++ b/examples/ministral/ministral-small-qlora.yaml @@ -0,0 +1,67 @@ +base_model: mistralai/Ministral-8B-Instruct-2410 + +# Enable to use mistral-common tokenizer +tokenizer_use_mistral_common: true + +# Automatically upload checkpoint and final model to HF +# hub_model_id: username/custom_model_name + +plugins: + - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin + +load_in_8bit: false +load_in_4bit: true + +datasets: + - path: fozziethebeat/alpaca_messages_2k_test + type: chat_template + +dataset_prepared_path: last_run_prepared +val_set_size: 0.1 +output_dir: ./outputs/lora-out + +adapter: qlora +lora_model_dir: + +sequence_len: 2048 +sample_packing: true + +lora_r: 32 +lora_alpha: 16 +lora_dropout: 0.05 +lora_target_linear: true +lora_target_modules: + - gate_proj + - down_proj + - up_proj + - q_proj + - v_proj + - k_proj + - o_proj + +wandb_project: +wandb_entity: +wandb_watch: +wandb_name: +wandb_log_model: + +gradient_accumulation_steps: 4 +micro_batch_size: 2 +num_epochs: 1 +optimizer: adamw_bnb_8bit +lr_scheduler: cosine +learning_rate: 0.0002 + +bf16: auto +tf32: false + +gradient_checkpointing: true +resume_from_checkpoint: +logging_steps: 1 +flash_attention: true + +warmup_ratio: 0.1 +evals_per_epoch: 1 +saves_per_epoch: 1 + +# save_first_step: true # uncomment this to validate checkpoint saving works with your config diff --git a/examples/ministral/think/README.md b/examples/ministral/think/README.md new file mode 100644 index 000000000..0ee5ea876 --- /dev/null +++ b/examples/ministral/think/README.md @@ -0,0 +1,99 @@ +# Ministral3 2512 Thinking Fine-tuning + +This guide covers fine-tuning [Ministral3 2512](https://huggingface.co/collections/mistralai/ministral-3) with thinking capabilities using Axolotl. The thinking model enables explicit Chain-of-Thought reasoning with separate thinking and response sections. + +Thanks to the team at MistralAI for giving us early access to prepare for these releases. + +## Prerequisites + +Before starting, ensure you have: +- Installed Axolotl (see [main README](../README.md)) + +## Getting Started + +1. Install transformers v5 + + ```bash + pip install transformers==5.0.0rc0 + ``` + + Note: This is still experimental in Axolotl. Other stuff may break. + +2. Upgrade `mistral-common` + + ```bash + pip install mistral-common==1.8.6 + ``` + +3. Swap to the Axolotl transformers v5 branch + + ```bash + # copy examples/ministral/think/ministral3-small-think-qlora.yaml somewhere + cp examples/ministral/think/ministral3-small-think-qlora.yaml ministral3-small-think-qlora.yaml + + git fetch + git checkout transformers-v5 + ``` + +4. Run the thinking model fine-tuning: + + ```bash + axolotl train ministral3-small-think-qlora.yaml + ``` + +This config uses about 4.76 GiB VRAM. + +### Tips + +- Dataset uses multi-content format with `type: thinking` support. See [Dataset Format](#dataset-format) below. +- You cannot mix `content: str` and `content: list[dict]`, otherwise, dataset loading will fail. Keep it consistent. + +## Dataset Format + +The thinking model requires the multi-content dataset format with support for an extra `role: thinking` within system and assistant messages. + +Example format: + +```json +{ + "messages": [ + { + "role": "system", + "content": [ + { "type": "text", "text": "{SYSTEM_PROMPT}"} + ] + }, + { + "role": "user", + "content": [ + { "type": "text", "text": "Solve this step by step: What is 15% of 240?"} + ] + }, + { + "role": "assistant", + "content": [ + { + "type": "thinking", + "thinking": "I need to calculate 15% of 240. First, I'll convert 15% to decimal: 0.15. Then multiply: 0.15 × 240 = 36." + }, + { + "type": "text", + "text": "To find 15% of 240, I'll multiply 240 by 0.15:\n\n240 × 0.15 = 36\n\nTherefore, 15% of 240 is 36." + } + ] + } + ] +} +``` + +### Advanced Options + +The `thinking` section supports an optional `closed` parameter: + +```json +{ + "type": "thinking", + "thinking": "Internal reasoning here...", + "closed": true // Default: true, controls adding the closing [/THINK] tag +} +``` diff --git a/examples/ministral/think/ministral3-small-think-qlora.yaml b/examples/ministral/think/ministral3-small-think-qlora.yaml new file mode 100644 index 000000000..987c0bd54 --- /dev/null +++ b/examples/ministral/think/ministral3-small-think-qlora.yaml @@ -0,0 +1,67 @@ +base_model: mistralai/Ministral-3-3B-Reasoning-2512 + +# Enable to use mistral-common tokenizer +tokenizer_use_mistral_common: true + +# Automatically upload checkpoint and final model to HF +# hub_model_id: username/custom_model_name + +plugins: + - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin + +load_in_8bit: false +load_in_4bit: true + +datasets: + - path: Nanobit/text-think-2k-test + type: chat_template + +dataset_prepared_path: last_run_prepared +val_set_size: 0 +output_dir: ./outputs/lora-out + +adapter: qlora +lora_model_dir: + +sequence_len: 2048 +sample_packing: true + +lora_r: 32 +lora_alpha: 16 +lora_dropout: 0.05 +lora_target_linear: true +lora_target_modules: + - gate_proj + - down_proj + - up_proj + - q_proj + - v_proj + - k_proj + - o_proj + +wandb_project: +wandb_entity: +wandb_watch: +wandb_name: +wandb_log_model: + +gradient_accumulation_steps: 4 +micro_batch_size: 2 +num_epochs: 1 +optimizer: adamw_bnb_8bit +lr_scheduler: cosine +learning_rate: 0.0002 + +bf16: auto +tf32: false + +gradient_checkpointing: true +resume_from_checkpoint: +logging_steps: 1 +flash_attention: true + +warmup_ratio: 0.1 +evals_per_epoch: 1 +saves_per_epoch: 1 + +# save_first_step: true # uncomment this to validate checkpoint saving works with your config diff --git a/examples/olmo3/README.md b/examples/olmo3/README.md index d4dbe05a9..2f98eb73e 100644 --- a/examples/olmo3/README.md +++ b/examples/olmo3/README.md @@ -6,24 +6,16 @@ This guide shows how to fine-tune it with Axolotl with multi-turn conversations ## Getting started -1. Install Axolotl following the [installation guide](https://docs.axolotl.ai/docs/installation.html). +1. Install Axolotl following the [installation guide](https://docs.axolotl.ai/docs/installation.html). + +2. Install [Cut Cross Entropy](https://docs.axolotl.ai/docs/custom_integrations.html#cut-cross-entropy) to reduce training VRAM usage. + +3. Run the finetuning example: - Here is an example of how to install from pip: ```bash - # Ensure you have a compatible version of Pytorch installed - pip3 install packaging setuptools wheel ninja - pip3 install --no-build-isolation 'axolotl[flash-attn]>=0.12.0' - - # Install Cut Cross Entropy - python scripts/cutcrossentropy_install.py | sh + axolotl train examples/olmo3/olmo3-7b-qlora.yaml ``` -2. Run the finetuning example: - -```bash -axolotl train examples/olmo3/olmo3-7b-qlora.yaml -``` - Let us know how it goes. Happy finetuning! 🚀 ### TIPS diff --git a/scripts/cutcrossentropy_install.py b/scripts/cutcrossentropy_install.py index 91d0f45d6..ec5c6d475 100644 --- a/scripts/cutcrossentropy_install.py +++ b/scripts/cutcrossentropy_install.py @@ -29,5 +29,5 @@ UV_PREFIX = "uv " if USE_UV else "" print( UNINSTALL_PREFIX - + f'{UV_PREFIX}pip install "cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@5eff953"' + + f'{UV_PREFIX}pip install "cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@f643b88"' ) diff --git a/src/axolotl/integrations/cut_cross_entropy/README.md b/src/axolotl/integrations/cut_cross_entropy/README.md index 4f98ac089..2c5b0f6e5 100644 --- a/src/axolotl/integrations/cut_cross_entropy/README.md +++ b/src/axolotl/integrations/cut_cross_entropy/README.md @@ -19,7 +19,7 @@ python scripts/cutcrossentropy_install.py | sh - If you are installing from pip ```bash -pip3 uninstall -y cut-cross-entropy && pip3 install "cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@5eff953" +pip3 uninstall -y cut-cross-entropy && pip3 install "cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@f643b88" ``` ## Usage @@ -61,6 +61,8 @@ plugins: - llama4 - llama4_text - llava +- ministral +- ministral3 - mistral - mistral3 - mixtral diff --git a/src/axolotl/integrations/cut_cross_entropy/__init__.py b/src/axolotl/integrations/cut_cross_entropy/__init__.py index b8f7e9da3..98a1659b1 100644 --- a/src/axolotl/integrations/cut_cross_entropy/__init__.py +++ b/src/axolotl/integrations/cut_cross_entropy/__init__.py @@ -35,7 +35,7 @@ LOG = get_logger(__name__) _CCE_INSTALL_MESSAGE = ( "Please install Axolotl's fork of cut_cross_entropy with transformers support using " - '`pip install "cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@5eff953"`' + '`pip install "cut-cross-entropy[transformers] @ git+https://github.com/axolotl-ai-cloud/ml-cross-entropy.git@f643b88"`' ) diff --git a/src/axolotl/monkeypatch/multipack.py b/src/axolotl/monkeypatch/multipack.py index 9642b1edb..6a6b935be 100644 --- a/src/axolotl/monkeypatch/multipack.py +++ b/src/axolotl/monkeypatch/multipack.py @@ -52,6 +52,8 @@ SUPPORTED_MULTIPACK_MODEL_TYPES = [ "olmo", "olmo2", "olmo3", + "ministral", + "ministral3", "afmoe", ] diff --git a/src/axolotl/utils/mistral/mistral_tokenizer.py b/src/axolotl/utils/mistral/mistral_tokenizer.py index 0414ece78..af174cdac 100644 --- a/src/axolotl/utils/mistral/mistral_tokenizer.py +++ b/src/axolotl/utils/mistral/mistral_tokenizer.py @@ -218,3 +218,10 @@ class HFMistralTokenizer(MistralCommonTokenizer): model_input_names=model_input_names, clean_up_tokenization_spaces=clean_up_tokenization_spaces, ) + + def save_pretrained(self, *args, **kwargs) -> tuple[str, ...]: + """ + Patches to remove save_jinja_files from being passed onwards. + """ + kwargs.pop("save_jinja_files", None) + return super().save_pretrained(*args, **kwargs)