tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Mixtral official (#942)

Browse Source 
* multipack support for official mixtral implementation

* fix patch to load multipack for mixtral

* chore: lint
This commit is contained in:
Wing Lian
2023-12-11 23:44:33 -05:00
committed by GitHub
parent 9a5eb3990c
commit 7fabc4d95e
8 changed files with 443 additions and 1703 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
examples/mistral/mixtral.yml
View File

@@ -1,5 +1,5 @@
base_model: DiscoResearch/mixtral-7b-8expert
model_type: MixtralForCausalLM
base_model: mistralai/Mixtral-8x7B-v0.1
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true