4f5e8a328a
* feat: add xiaomi's mimo 7b * fix: pin revision * fix: update trinity docs and pin revision * fix: wrong config name * feat: add vram usage * feat: add plano * feat: update plano vram usage * chore: comments
Finetune ArceeAI's Trinity with Axolotl
Trinity is a family of open weight MoE models trained by Arcee.ai.
This guide shows how to fine-tune it with Axolotl with multi-turn conversations and proper masking.
Getting started
-
Install Axolotl following the main from the installation guide.
-
Run the finetuning example:
axolotl train examples/trinity/trinity-nano-preview-qlora.yaml
This config uses about 24.9 GiB VRAM.
Let us know how it goes. Happy finetuning! 🚀
TIPS
- For inference, the official Arcee.ai team recommends
top_p: 0.75,temperature: 0.15,top_k: 50, andmin_p: 0.06. - You can run a full finetuning by removing the
adapter: qloraandload_in_4bit: truefrom the config. - Read more on how to load your own dataset at docs.
- The dataset format follows the OpenAI Messages format as seen here.
Optimization Guides
Please check the Optimizations doc.
Limitations
Cut Cross Entropy (CCE): Currently not supported. We plan to include CCE support for Trinity in the near future.