Files

Wing Lian 814aee6603 Phi2 multipack (#1173 )

* phi2 multipack

* update validation and examples for phi

* more updates to phi examples

* make sure to use the correct collator for phi multipack

* phi needs attention mask now for multipack

* if the special token already exists in the tokenizer, don't require in lora modules to save

* fix qlora yml for phi, fix phi test validation

* test qlora too

* make sure flash attention is enabled for the test

* don't use remote code for phi anymore

* reduce sequence len for sample packing phi

2024-01-23 12:54:36 -05:00

phi2-ft.yml

Phi2 multipack (#1173 )

2024-01-23 12:54:36 -05:00

phi-ft.yml

Phi2 multipack (#1173 )

2024-01-23 12:54:36 -05:00

phi-qlora.yml

Phi2 multipack (#1173 )

2024-01-23 12:54:36 -05:00

README.md

make phi training work with Loras (#588 )

2023-09-15 20:51:55 -04:00

README.md

Phi

Due to some nuances with the phi code, please use deepspeed when training phi for full finetune.

accelerate launch -m axolotl.cli.train examples/phi/phi-ft.yml --deepspeed deepspeed/zero1.json

# OR

python -m axolotl.cli.train examples/phi/phi-qlora.yml