added tiny llama examples for lora and qlora (#1027)

* added tiny llama examples for lora and qlora

* corrected yml files and removed tiny-llama.yml from llama-2 example
This commit is contained in:
Tim Dolan
2024-01-02 20:00:37 -05:00
committed by GitHub
parent 4d2e842e46
commit c75f916745
3 changed files with 87 additions and 6 deletions

View File

@@ -0,0 +1,17 @@
# Overview
This is a simple example of how to finetune TinyLlama1.1B using either lora or qlora:
LoRa:
```
accelerate launch -m axolotl.cli.train examples/tiny-llama/lora.yml
```
qLoRa:
```
accelerate launch -m axolotl.cli.train examples/tiny-llama/qlora.yml
```
Both take about 10 minutes to complete on a 4090.