axolotl

Files

Maxime 0f6af36d50 Mps mistral lora (#1292 ) [skip ci]

* Lora example for Mistral on MPS backend

* Add some MPS documentation

* Update examples/mistral/lora-mps.yml

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>

* Update examples/mistral/lora-mps.yml

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>

* Update README.md

---------

Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-02-26 22:39:57 -05:00

Mistral-7b-example

fix(examples): remove is_*_derived as it's parsed automatically (#1297 )

2024-02-22 00:52:46 +09:00

config.yml

fix(examples): remove is_*_derived as it's parsed automatically (#1297 )

2024-02-22 00:52:46 +09:00

lora-mps.yml

Mps mistral lora (#1292 ) [skip ci]

2024-02-26 22:39:57 -05:00

mixtral.yml

Add seq2seq eval benchmark callback (#1274 )

2024-02-13 08:24:30 -08:00

qlora.yml

fix(examples): remove is_*_derived as it's parsed automatically (#1297 )

2024-02-22 00:52:46 +09:00

README.md

Mixtral fixes 20240124 (#1192 ) [skip ci]

2024-01-24 14:59:57 -05:00

README.md

Mistral 7B is a language model with a total of 7.3 billion parameters, showcasing a notable performance across a variety of benchmarks.

Fine Tune:

accelerate launch -m axolotl.cli.train examples/mistral/config.yml

If you run into CUDA OOM, use deepspeed with config zero2.json:

accelerate launch -m axolotl.cli.train examples/mistral/config.yml --deepspeed deepspeed_configs/zero2.json