tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Add a config not to shuffle merged dataset (#1394) [skip ci]

Browse Source 
* Add a config not to shuffle merged dataset

* Update README.md

* Update src/axolotl/utils/config/models/input/v0_4_1/__init__.py

Co-authored-by: Wing Lian <wing.lian@gmail.com>

* invert the condition name

* update README

* info -> debug

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>
This commit is contained in:
Seungduk Kim
2024-03-19 20:51:00 +09:00
committed by GitHub
parent b1e3e1b25f
commit 43bdc5d3de
3 changed files with 15 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -678,6 +678,10 @@ datasets:
# For `completion` datsets only, uses the provided field instead of `text` column
field:
# If false, the datasets will not be shuffled and will keep their original order in `datasets`.
# The same applies to the `test_datasets` option and the `pretraining_dataset` option. Default is true.
shuffle_merged_datasets: true
# A list of one or more datasets to eval the model with.
# You can use either test_datasets, or val_set_size, but not both.
test_datasets: