Files

Leonardo Emili 5a5d47458d Add seq2seq eval benchmark callback (#1274 )

* Add CausalLMBenchEvalCallback for measuring seq2seq performance

* Fix code for pre-commit

* Fix typing and improve logging

* eval_sample_packing must be false with CausalLMBenchEvalCallback

2024-02-13 08:24:30 -08:00

qlora.yml

Add seq2seq eval benchmark callback (#1274 )

2024-02-13 08:24:30 -08:00

README.md

Add an example config for finetuning a 34B model on a 24GB GPU (#1000 )

2023-12-25 10:29:55 -08:00

README.md

Overview

This is an example of a Yi-34B-Chat configuration. It demonstrates that it is possible to finetune a 34B model on a GPU with 24GB of VRAM.

Tested on an RTX 4090 with python -m axolotl.cli.train examples/mistral/qlora.yml, a single epoch of finetuning on the alpaca dataset using qlora runs in 47 mins, using 97% of available memory.