diff --git a/examples/mistral4/README.md b/examples/mistral4/README.md index 651384791..3151069ba 100644 --- a/examples/mistral4/README.md +++ b/examples/mistral4/README.md @@ -6,9 +6,6 @@ Thanks to the team at MistralAI for giving us early access to prepare for this r ## Getting started -Note: Training this model requires weights in BF16 which we will link to later. -Users interested in training can convert / descale the existing FP8 weights. - 1. Install Axolotl following the [installation guide](https://docs.axolotl.ai/docs/installation.html). 2. Install [Cut Cross Entropy](https://docs.axolotl.ai/docs/custom_integrations.html#cut-cross-entropy) to reduce training VRAM usage diff --git a/examples/mistral4/fft-text.yml b/examples/mistral4/fft-text.yml index e01d96dad..3acb5b2ed 100644 --- a/examples/mistral4/fft-text.yml +++ b/examples/mistral4/fft-text.yml @@ -1,4 +1,4 @@ -base_model: mistralai/Mistral-Small-4-119B-2603 +base_model: axolotl-ai-co/Mistral-Small-4-119B-2603-BF16 plugins: - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin diff --git a/examples/mistral4/fft-vision.yml b/examples/mistral4/fft-vision.yml index aa65dfa6d..baff37fe4 100644 --- a/examples/mistral4/fft-vision.yml +++ b/examples/mistral4/fft-vision.yml @@ -1,4 +1,4 @@ -base_model: mistralai/Mistral-Small-4-119B-2603 +base_model: axolotl-ai-co/Mistral-Small-4-119B-2603-BF16 processor_type: AutoProcessor plugins: diff --git a/examples/mistral4/qlora-text.yml b/examples/mistral4/qlora-text.yml index ed38053f6..ae0cdcead 100644 --- a/examples/mistral4/qlora-text.yml +++ b/examples/mistral4/qlora-text.yml @@ -1,4 +1,4 @@ -base_model: mistralai/Mistral-Small-4-119B-2603 +base_model: axolotl-ai-co/Mistral-Small-4-119B-2603-BF16 plugins: - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin diff --git a/examples/mistral4/qlora-vision.yml b/examples/mistral4/qlora-vision.yml index 95b8138ce..a80d166dd 100644 --- a/examples/mistral4/qlora-vision.yml +++ b/examples/mistral4/qlora-vision.yml @@ -1,4 +1,4 @@ -base_model: mistralai/Mistral-Small-4-119B-2603 +base_model: axolotl-ai-co/Mistral-Small-4-119B-2603-BF16 processor_type: AutoProcessor plugins: