Files

Wing Lian 54d2ac155b Mixtral fixes 20240124 (#1192 ) [skip ci]

* mixtral nccl fixes

* make sure to patch for z3

2024-01-24 14:59:57 -05:00

phi2-ft.yml

Phi2 multipack (#1173 )

2024-01-23 12:54:36 -05:00

phi-ft.yml

Phi2 multipack (#1173 )

2024-01-23 12:54:36 -05:00

phi-qlora.yml

Phi2 multipack (#1173 )

2024-01-23 12:54:36 -05:00

README.md

Mixtral fixes 20240124 (#1192 ) [skip ci]

2024-01-24 14:59:57 -05:00

README.md

Phi

Due to some nuances with the phi code, please use deepspeed when training phi for full finetune.

accelerate launch -m axolotl.cli.train examples/phi/phi-ft.yml --deepspeed deepspeed_configs/zero1.json

# OR

python -m axolotl.cli.train examples/phi/phi-qlora.yml