axolotl

Author	SHA1	Message	Date
Wing Lian	22810c97b7	use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci] * use warmup_ratio as a better default than warmup steps since it's data dependent * replace remainder of warmup_steps	2025-07-30 06:44:06 -04:00
Wing Lian	af8d257aa2	make pad_to_sequence_len default to the same value as sample_packing (#2941 ) [skip ci] * make pad_to_sequence_len default to the same value as sample_packing * remove duplicate validation * fix test * update description meta Co-authored-by: NanoCode012 <nano@axolotl.ai> --------- Co-authored-by: NanoCode012 <nano@axolotl.ai>	2025-07-21 11:40:56 -04:00
Dan Saunders	10ba1622f7	checkpoint model on first step callback (#2906 ) * checkpoint model on first step callback * remove debug * add test cases; update existing tests not to save on first step * move test out of solo * delete * default to False * typo	2025-07-15 15:00:48 -04:00
Wing Lian	dd8bad06d0	remove strict=false from example yamls [skip ci] (#2523 ) [skip ci]	2025-04-12 07:25:11 -07:00
Wing Lian	9f824ef76a	simplify the example configs to be more minimal and less daunting (#2486 ) [skip ci] * simplify the example configs to be more minimal and less daunting * drop empty s2_attention from example yamls	2025-04-04 13:47:26 -04:00
Sunny Liu	1c14c4a15c	Add hub model id config options to all example yml files (#2196 ) [skip ci] * added hub model_id in example yml * add hub model id to example yml	2024-12-17 11:24:30 -05:00
Wing Lian	e4063d60a7	bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 ) * bump transformers and set roundup_power2_divisions for more VRAM improvements * support for low bit optimizers from torch ao * fix check for alternate optimizers and use nous models on hf for llama3 * add missing check for ao_adamw_fp8 * fix check when using custom optimizers w adamw	2024-07-19 00:47:07 -04:00
RodriMora	219cd0d3c5	Fix eval_sample_packing in llama-3 lora example (#1716 ) [skip ci] * Fix eval_sample_packing in llama-3 lora example * Update examples/llama-3/lora-8b.yml Co-authored-by: Wing Lian <wing.lian@gmail.com> --------- Co-authored-by: Wing Lian <wing.lian@gmail.com>	2024-07-13 14:34:44 -04:00
Wing Lian	f5febc729a	fix lint issue that snuck through (#1665 )	2024-05-28 11:36:50 -04:00
Faria Huq	230e0ac363	Fix Lora config error for Llama3 (#1659 ) The current yml code throws an error: ValueError: Please set lora_modules_to_save to [`embed_tokens`, `lm_head`] when using an adapter and changing the special tokens. I added the required changes to resolve it	2024-05-28 11:25:08 -04:00
Wing Lian	4fde300e5f	update outputs path so that we can mount workspace to /workspace/data (#1623 ) * update outputs path so that we can mount workspace to /workspace/data * fix ln order	2024-05-15 12:44:13 -04:00
NanoCode012	0e8f340945	fix(yml): update llama-3 config (#1543 ) [skip ci]	2024-04-19 20:44:46 +09:00
Wing Lian	15f7910d33	llama-3 examples (#1537 )	2024-04-18 14:28:03 -04:00

13 Commits