axolotl

Files

Kevin Sydney 384b817dc0 Set eval_sample_packing to false in mistral config.yaml (#1003 )

Without eval_sampling_packing set to false, ValueError occurs with eval dataset split is too small for sample_packing.

2023-12-27 16:11:55 -08:00

config.yml

2023-12-27 16:11:55 -08:00

mixtral.yml

2023-12-22 12:57:02 -05:00

qlora.yml

change val size (#992 )

2023-12-22 16:18:16 +01:00

README.md

2023-09-28 10:24:56 -04:00

Mistral 7B is a language model with a total of 7.3 billion parameters, showcasing a notable performance across a variety of benchmarks.

Fine Tune:

accelerate launch -m axolotl.cli.train examples/mistral/config.yml

If you run into CUDA OOM, use deepspeed with config zero2.json:

accelerate launch -m axolotl.cli.train examples/mistral/config.yml --deepspeed deepspeed/zero2.json