don't train if eval split is too small (#873)

* allow zero len dataset

* better handling and warning of small eval splits

* raise error if eval split is too small

* don't mess with calculating total num steps in distributed context

* fix eval_sample_packing training args logic
This commit is contained in:
Wing Lian
2023-11-16 11:35:42 -05:00
committed by GitHub
parent 0de1457189
commit 797f3dd1de
4 changed files with 19 additions and 6 deletions

View File

@@ -182,7 +182,7 @@ class MultipackBatchSampler(BatchSampler):
# shave off 1% + 1 for dealing with variance in packing from random sampler to sampler
return max(
1,
0,
(
world_size
* math.floor(