Files

Tim Dolan c75f916745 added tiny llama examples for lora and qlora (#1027 )

* added tiny llama examples for lora and qlora

* corrected yml files and removed tiny-llama.yml from llama-2 example

2024-01-02 20:00:37 -05:00

Overview

This is a simple example of how to finetune TinyLlama1.1B using either lora or qlora:

LoRa:

accelerate launch -m axolotl.cli.train examples/tiny-llama/lora.yml

qLoRa:

accelerate launch -m axolotl.cli.train examples/tiny-llama/qlora.yml

Both take about 10 minutes to complete on a 4090.