axolotl

Files

Leonardo Emili 5a5d47458d Add seq2seq eval benchmark callback (#1274 )

* Add CausalLMBenchEvalCallback for measuring seq2seq performance

* Fix code for pre-commit

* Fix typing and improve logging

* eval_sample_packing must be false with CausalLMBenchEvalCallback

2024-02-13 08:24:30 -08:00

Mistral-7b-example

Add seq2seq eval benchmark callback (#1274 )

2024-02-13 08:24:30 -08:00

config.yml

Add seq2seq eval benchmark callback (#1274 )

2024-02-13 08:24:30 -08:00

mixtral.yml

Add seq2seq eval benchmark callback (#1274 )

2024-02-13 08:24:30 -08:00

qlora.yml

Add seq2seq eval benchmark callback (#1274 )

2024-02-13 08:24:30 -08:00

README.md

Mixtral fixes 20240124 (#1192 ) [skip ci]

2024-01-24 14:59:57 -05:00

README.md

Mistral 7B is a language model with a total of 7.3 billion parameters, showcasing a notable performance across a variety of benchmarks.

Fine Tune:

accelerate launch -m axolotl.cli.train examples/mistral/config.yml

If you run into CUDA OOM, use deepspeed with config zero2.json:

accelerate launch -m axolotl.cli.train examples/mistral/config.yml --deepspeed deepspeed_configs/zero2.json