tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Fix(doc): Clarify doc on attention configs and missing pad_token (#2455) [skip ci]

Browse Source 
* fix: clarify input type

* fix: handling of error message if data_files not available

* fix: clarify attention handling

* fix: add doc on missing pad token
This commit is contained in:
NanoCode012
2025-04-01 02:47:28 +07:00
committed by GitHub
parent b6fc46ada8
commit 7acf93b59f
3 changed files with 29 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
src/axolotl/utils/data/shared.py
View File

@@ -238,7 +238,8 @@ def load_dataset_w_config(
trust_remote_code=config_dataset.trust_remote_code,
**load_ds_kwargs,
)
else:
elif config_dataset.data_files:
fp: str | list[str] | None = None
if isinstance(config_dataset.data_files, str):
fp = hf_hub_download(
repo_id=config_dataset.path,