Wing Lian 71d4030b79 gradient accumulation tests, embeddings w pad_token fix, smaller models (#2059) ...

* add more test cases for gradient accumulation and fix zero3

* swap out for smaller model

* fix missing return

* fix missing pad_token in config

* support concurrency for multigpu testing

* cast empty deepspeed to empty string for zero3 check

* fix temp_dir as fixture so parametrize works properly

* fix test file for multigpu evals

* don't use default

* don't use default for fsdp_state_dict_type

* don't use llama tokenizer w smollm

* also automatically cancel multigpu for concurrency

2024-11-14 12:59:00 -05:00

2.2 KiB

Raw Blame History

View Raw