Files

Maxime fac2d98c26 Add MPS support (#1264 )

* add mps support

* linter stuff

* CI fixes

* install packaging for various tests

* Update setup.py

* Revert "install packaging for various tests"

This reverts commit 980e7aa44d.

* Revert "CI fixes"

This reverts commit 4609e3b166.

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-02-12 08:30:32 -05:00

lora-mps.yml

Add MPS support (#1264 )

2024-02-12 08:30:32 -05:00

lora.yml

set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122 ) [skip ci]

2024-01-22 18:44:01 -05:00

pretrain.yml

Pretrain transforms (#1261 )

2024-02-06 00:37:03 -05:00

qlora.yml

set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122 ) [skip ci]

2024-01-22 18:44:01 -05:00

README.md

added tiny llama examples for lora and qlora (#1027 )

2024-01-02 20:00:37 -05:00

README.md

Overview

This is a simple example of how to finetune TinyLlama1.1B using either lora or qlora:

LoRa:

accelerate launch -m axolotl.cli.train examples/tiny-llama/lora.yml

qLoRa:

accelerate launch -m axolotl.cli.train examples/tiny-llama/qlora.yml

Both take about 10 minutes to complete on a 4090.