Teknium d3193beac3 Fix Deepspeed Zero3 Config (#791) ...

* Update zero3.json

Take away CPU Offload by default (Slows things down horribly, better off reducing batchsize), and changes LR Scheduler to a properly decaying one

* Update zero3.json

fix something

2023-10-27 21:57:02 -04:00

..

zero1.json

update README w deepspeed info (#605)

2023-09-22 00:15:52 -04:00

zero2.json

update README w deepspeed info (#605)

2023-09-22 00:15:52 -04:00

zero3.json

Fix Deepspeed Zero3 Config (#791)

2023-10-27 21:57:02 -04:00