tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

make phi training work with Loras (#588)

Browse Source 
* valdiation for phi loras

* fix model config class check

* update readme for phi traiing
This commit is contained in:
Wing Lian
2023-09-15 20:51:55 -04:00
committed by GitHub
parent be75668400
commit 62eaee7649
4 changed files with 114 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
examples/phi/README.md
View File

@@ -1,7 +1,11 @@
# Phi
Due to some nuances with the phi code, please use deepspeed when training phi.
Due to some nuances with the phi code, please use deepspeed when training phi for full finetune.
```shell
accelerate launch scripts/finetune.py examples/phi/phi-ft.yml --deepspeed deepspeed/zero1.json
accelerate launch -m axolotl.cli.train examples/phi/phi-ft.yml --deepspeed deepspeed/zero1.json
# OR
python -m axolotl.cli.train examples/phi/phi-qlora.yml
```