Wing Lian 54d2ac155b Mixtral fixes 20240124 (#1192) [skip ci] ...

* mixtral nccl fixes

* make sure to patch for z3

2024-01-24 14:59:57 -05:00

..

zero1.json

Mixtral fixes 20240124 (#1192) [skip ci]

2024-01-24 14:59:57 -05:00

zero2.json

Mixtral fixes 20240124 (#1192) [skip ci]

2024-01-24 14:59:57 -05:00

zero3_bf16.json

Mixtral fixes 20240124 (#1192) [skip ci]

2024-01-24 14:59:57 -05:00

zero3.json

Mixtral fixes 20240124 (#1192) [skip ci]

2024-01-24 14:59:57 -05:00