3 Commits

Author	SHA1	Message	Date
JinK	553c80f79a	streaming multipack for pretraining dataset (#959) ... * [Feat] streaming multipack * WIP make continued pretraining work w multipack * fix up hadrcoding, lint * fix dict check * update test for updated pretraining multipack code * fix hardcoded data collator fix for multipack pretraining * fix the collator to be the max length for multipack pretraining * don't bother with latest tag for test * cleanup docker build/test --------- Co-authored-by: jinwonkim93@github.com <jinwonkim> Co-authored-by: Wing Lian <wing.lian@gmail.com>	2024-01-05 22:13:21 -05:00
NanoCode012	8ba27f3bde	fix: lint (#1037)	2024-01-03 10:23:44 -05:00
Tim Dolan	c75f916745	added tiny llama examples for lora and qlora (#1027) ... * added tiny llama examples for lora and qlora * corrected yml files and removed tiny-llama.yml from llama-2 example	2024-01-02 20:00:37 -05:00