added tiny llama examples for lora and qlora (#1027)
* added tiny llama examples for lora and qlora * corrected yml files and removed tiny-llama.yml from llama-2 example
This commit is contained in:
17
examples/tiny-llama/README.md
Normal file
17
examples/tiny-llama/README.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Overview
|
||||
|
||||
This is a simple example of how to finetune TinyLlama1.1B using either lora or qlora:
|
||||
|
||||
LoRa:
|
||||
|
||||
```
|
||||
accelerate launch -m axolotl.cli.train examples/tiny-llama/lora.yml
|
||||
```
|
||||
|
||||
qLoRa:
|
||||
|
||||
```
|
||||
accelerate launch -m axolotl.cli.train examples/tiny-llama/qlora.yml
|
||||
```
|
||||
|
||||
Both take about 10 minutes to complete on a 4090.
|
||||
Reference in New Issue
Block a user