Files

Wing Lian 782b6a4216 set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122 ) [skip ci]

* set fp16 to false if bf16, update bf16: auto in example YAMLs

* unset fp16 so that it fallsback properly if bf16 isn't available

* Update README.md [skip-ci]

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>

* test that bf16 disables fp16

---------

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>

2024-01-22 18:44:01 -05:00

qlora.yml

set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122 ) [skip ci]

2024-01-22 18:44:01 -05:00

README.md

Add an example config for finetuning a 34B model on a 24GB GPU (#1000 )

2023-12-25 10:29:55 -08:00

README.md

Overview

This is an example of a Yi-34B-Chat configuration. It demonstrates that it is possible to finetune a 34B model on a GPU with 24GB of VRAM.

Tested on an RTX 4090 with python -m axolotl.cli.train examples/mistral/qlora.yml, a single epoch of finetuning on the alpaca dataset using qlora runs in 47 mins, using 97% of available memory.