tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Files
02af0820f764e862d27ef18b3f1f4dea3c35699c
axolotl/examples/jamba/README.md
Wing Lian 02af0820f7 Jamba (#1451) 
* fixes for larger models

* add qlora example for deepspeed

* add readme for jamba
2024-03-28 21:03:22 -04:00

156 B
Raw Blame History

Jamba

qlora w/ deepspeed needs at least 2x GPUs and 35GiB VRAM per GPU

qlora single-gpu - training will start, but loss is off by an order of magnitude

Reference in New Issue View Git Blame Copy Permalink