Wing Lian 5ea3aa31f0 Fix Deepspeed loading (#950) ...

* add check for zero3

* freeze parameters

* fixes for deepspeed loading

* fix model parameter check

* unfrozen parameters in example mixtral and logging when unfreezing

2023-12-13 16:03:23 -05:00

..

zero1.json

Remove learning rate scheduler in deepspeed config to avoid conflict (#909)

2023-12-04 05:17:38 -05:00

zero2.json

Remove learning rate scheduler in deepspeed config to avoid conflict (#909)

2023-12-04 05:17:38 -05:00

zero3_bf16.json

Fix Deepspeed loading (#950)

2023-12-13 16:03:23 -05:00

zero3.json

Remove learning rate scheduler in deepspeed config to avoid conflict (#909)

2023-12-04 05:17:38 -05:00