tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

qwen2_moe support w multipack (#1455)

Browse Source
This commit is contained in:
Wing Lian
2024-03-29 11:04:53 -04:00
committed by GitHub
parent 4a92a3b9ee
commit 6086be85f7
6 changed files with 147 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
examples/qwen/README.md Normal file
View File

@@ -0,0 +1,10 @@
# Qwen
TODO
# Qwen2 MoE
✅ multipack
✅ qwen2_moe 4-bit QLoRA
✅ qwen2_moe 16-bit LoRA
❓ qwen2_moe 8-bit LoRA