3 Commits

Author	SHA1	Message	Date
Sunny Liu	1c14c4a15c	Add hub model id config options to all example yml files (#2196) [skip ci] ... * added hub model_id in example yml * add hub model id to example yml	2024-12-17 11:24:30 -05:00
Wing Lian	3ebf22464b	qlora-fsdp ram efficient loading with hf trainer (#1791) ... * fix 405b with lower cpu ram requirements * make sure to use doouble quant and only skip output embeddings * set model attributes * more fixes for sharded fsdp loading * update the base model in example to use pre-quantized nf4-bf16 weights * upstream fixes for qlora+fsdp	2024-07-30 19:21:38 -04:00
Wing Lian	94ba93259f	various batch of fixes (#1785) ... * various batch of fixes * more tweaks * fix autoawq requirement for torch flexibility * simplify conditionals * multi-node fixes wip * bump transformers and include 405b qlora+fsdp yaml	2024-07-28 07:25:54 -04:00