2 Commits

Author	SHA1	Message	Date
Wing Lian	a1790f2652	replace tensorboard checks with helper function (#2120) [skip ci] ... * replace tensorboard checks with helper function * move helper function * use relative	2024-12-03 21:06:20 -05:00
Wing Lian	1cf7075d18	support seperate lr for embeddings, similar to loraplus (#1910) [skip ci] ... * support seperate lr for embeddings, similar to loraplus * add test case for train w lr embedding scale * use kwarg for optimizer * make sure to handle the optimizer creation * make sure to handle for embedding_lr too * use smollm for e2e, check for embeddings lr first before wdecay	2024-11-29 20:38:20 -05:00