tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Mixtral official (#942)

Browse Source 
* multipack support for official mixtral implementation

* fix patch to load multipack for mixtral

* chore: lint
This commit is contained in:
Wing Lian
2023-12-11 23:44:33 -05:00
committed by GitHub
parent 9a5eb3990c
commit 7fabc4d95e
8 changed files with 443 additions and 1703 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
requirements.txt
View File

@@ -2,7 +2,7 @@
auto-gptq==0.5.1
packaging
peft==0.6.0
transformers @ git+https://github.com/huggingface/transformers.git@df5c5c62ae253055336f5bb0828ca8e3e15ab6bd
transformers @ git+https://github.com/huggingface/transformers.git@e5079b0b2abcef11ecbdae60ba4a6636c57b725d
tokenizers==0.15.0
bitsandbytes>=0.41.1
accelerate==0.24.1