tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Add desc to map/filter (#1162)

Browse Source 
* Add desc to map/filter

* update descriptions

---------

Co-authored-by: Wing Lian <wing.lian@gmail.com>
This commit is contained in:
Casper
2024-01-23 03:30:53 +01:00
committed by GitHub
parent cda52dc32b
commit 684038111e
4 changed files with 16 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
src/axolotl/cli/__init__.py
View File

@@ -410,7 +410,10 @@ def load_rl_datasets(
for i, data_set in enumerate(train_datasets):
_type = cfg.datasets[i]["type"]
ds_type_fn = locals()[_type]
train_datasets[i] = data_set.map(ds_type_fn)
train_datasets[i] = data_set.map(
ds_type_fn,
desc="Mapping RL Dataset",
)
train_dataset = concatenate_datasets(train_datasets)
# eval_dataset = eval_dataset.map(intel_apply_chatml)