af8d257aa2
* make pad_to_sequence_len default to the same value as sample_packing * remove duplicate validation * fix test * update description meta Co-authored-by: NanoCode012 <nano@axolotl.ai> --------- Co-authored-by: NanoCode012 <nano@axolotl.ai>
Overview
This is a simple example of how to finetune TinyLlama1.1B using either lora or qlora:
LoRa:
accelerate launch -m axolotl.cli.train examples/tiny-llama/lora.yml
qLoRa:
accelerate launch -m axolotl.cli.train examples/tiny-llama/qlora.yml
Both take about 10 minutes to complete on a 4090.