c75f916745
* added tiny llama examples for lora and qlora * corrected yml files and removed tiny-llama.yml from llama-2 example
18 lines
319 B
Markdown
18 lines
319 B
Markdown
# Overview
|
|
|
|
This is a simple example of how to finetune TinyLlama1.1B using either lora or qlora:
|
|
|
|
LoRa:
|
|
|
|
```
|
|
accelerate launch -m axolotl.cli.train examples/tiny-llama/lora.yml
|
|
```
|
|
|
|
qLoRa:
|
|
|
|
```
|
|
accelerate launch -m axolotl.cli.train examples/tiny-llama/qlora.yml
|
|
```
|
|
|
|
Both take about 10 minutes to complete on a 4090.
|