It should be `qlora` instead of `lora`
TODO
✅ multipack ✅ qwen2_moe 4-bit QLoRA ✅ qwen2_moe 16-bit LoRA ❓ qwen2_moe 8-bit LoRA
