tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

pad_to_worst_case_seq_len boolean, for testing memory limits (#498)

Browse Source 
* pad_to_worst_case_seq_len boolean, for testing memory limits

* remove collator_pad_to_longest option since it does nothing

see docs: https://huggingface.co/docs/transformers/main_classes/data_collator#transformers.DataCollatorWithPadding.padding

True and "longest" mean the same thing

* rename to `pad_to_sequence_len, and ensure 64 alignment

---------

Co-authored-by: Aman Karmani <aman@tmm1.net>
This commit is contained in:
Birch-san
2023-08-28 23:47:16 +01:00
committed by GitHub
parent 267b7b24e5
commit 8e197f6fb4
3 changed files with 6 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/pythia-12b/config.yml
View File

@@ -47,4 +47,3 @@ local_rank:
gradient_checkpointing: true
fsdp:
fsdp_config:
collator_pad_to_longest: true