Files

Birch-san 8e197f6fb4 pad_to_worst_case_seq_len boolean, for testing memory limits (#498 )

* pad_to_worst_case_seq_len boolean, for testing memory limits

* remove collator_pad_to_longest option since it does nothing

see docs: https://huggingface.co/docs/transformers/main_classes/data_collator#transformers.DataCollatorWithPadding.padding

True and "longest" mean the same thing

* rename to `pad_to_sequence_len, and ensure 64 alignment

---------

Co-authored-by: Aman Karmani <aman@tmm1.net>

2023-08-28 18:47:16 -04:00

config.yml

pad_to_worst_case_seq_len boolean, for testing memory limits (#498 )

2023-08-28 18:47:16 -04:00

README.md

address PR feedback

2023-06-10 14:23:56 -04:00

README.md

Pythia 12B

Single-GPU A100 only (?)

python scripts/finetune.py examples/pythia-12b/config.yml

⚠️ Multiple-GPU A100 - Doesn't seem to work with multi-gpu without causing OOM! ⚠️