NanoCode012 946b497c3f feat: add deepspeed 3 with cpuoffload (#1466) ...

* feat: add deepspeed 3 with cpuoffload

* make bf16 explicit, add param only offload variant

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>

2024-04-01 21:42:52 +09:00

..

zero1.json

Set gradient_clipping to auto in DeepSpeed configs (#1382) [skip ci]

2024-03-10 20:50:12 -04:00

zero2.json

Set gradient_clipping to auto in DeepSpeed configs (#1382) [skip ci]

2024-03-10 20:50:12 -04:00

zero3_bf16_cpuoffload_all.json

feat: add deepspeed 3 with cpuoffload (#1466)

2024-04-01 21:42:52 +09:00

zero3_bf16_cpuoffload_params.json

feat: add deepspeed 3 with cpuoffload (#1466)

2024-04-01 21:42:52 +09:00

zero3_bf16.json

Set gradient_clipping to auto in DeepSpeed configs (#1382) [skip ci]

2024-03-10 20:50:12 -04:00

zero3.json

Set gradient_clipping to auto in DeepSpeed configs (#1382) [skip ci]

2024-03-10 20:50:12 -04:00