tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

multipack len should use max, not min (#863)

Browse Source
This commit is contained in:
Wing Lian
2023-11-15 12:52:32 -05:00
committed by GitHub
parent db8a8afcba
commit 0c2a630326
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/axolotl/utils/samplers/multipack.py
View File

@@ -181,7 +181,7 @@ class MultipackBatchSampler(BatchSampler):
) )
# shave off 1% + 1 for dealing with variance in packing from random sampler to sampler # shave off 1% + 1 for dealing with variance in packing from random sampler to sampler
return min( return max(
1, 1,
( (
world_size world_size