Wing Lian 976f85195a fixes to accelerator so that iterable pretraining datasets work (#1759) ...

* fixes to accelerator so that iterable pretraining datasets work

* fix the pretraining test params

* split batches, not dispatch batches needs to be set

* update c4 datasets

* set epochs in pretrain config test

* need to set both split_batches and dispatch_batches to false for pretraining

* fix bool val in comment

2024-07-17 10:58:38 -04:00

2.8 KiB

Raw Blame History

View Raw