tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

Browse Source 
* make pad_to_sequence_len default to the same value as sample_packing

* remove duplicate validation

* fix test

* update description meta

Co-authored-by: NanoCode012 <nano@axolotl.ai>

---------

Co-authored-by: NanoCode012 <nano@axolotl.ai>
This commit is contained in:
Wing Lian
2025-07-21 11:40:56 -04:00
committed by GitHub
parent db5f6f4693
commit af8d257aa2
90 changed files with 109 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/axolotl/utils/schemas/config.py
View File

@@ -435,7 +435,7 @@ class AxolotlInputConfig(
pad_to_sequence_len: bool | None = Field(
default=None,
json_schema_extra={
"description": "Pad inputs so each step uses constant sized buffers. This will reduce memory fragmentation and may prevent OOMs, by re-using memory more efficiently"
"description": "Pad inputs so each step uses constant sized buffers. This will reduce memory fragmentation and may prevent OOMs, by re-using memory more efficiently. Defaults to True if `sample_packing` enabled"
},
)
curriculum_sampling: bool | None = Field(