tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Add an example config for finetuning a 34B model on a 24GB GPU (#1000)

Browse Source 
* Add an example config for finetuning a 34B model on a 24GB GPU

* Remore wandb project
This commit is contained in:
Evan Griffiths
2023-12-25 18:29:55 +00:00
committed by GitHub
parent 628b754824
commit 6ef46f8dca
2 changed files with 81 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
examples/yi-34B-chat/README.md Normal file
View File

@@ -0,0 +1,5 @@
# Overview
This is an example of a Yi-34B-Chat configuration. It demonstrates that it is possible to finetune a 34B model on a GPU with 24GB of VRAM.
Tested on an RTX 4090 with `python -m axolotl.cli.train examples/mistral/qlora.yml`, a single epoch of finetuning on the alpaca dataset using qlora runs in 47 mins, using 97% of available memory.