3 Commits

Author	SHA1	Message	Date
Wing Lian	9d5c95db6f	Add support for Accelerate CP, ND examples, and fix for parallel config w fsdp (#3019) ... * fix for parallelism config from trainer * fix handling of parallelism_config w accelerate * add todo for removal * update to latest axolotl-contribs-mit for optimizer fix too * synchronize training after checkpoint save * dir spelling * use latest accelerate main * fix to not use partial state parallelism_config * more fixeS * use most recent accelerate fix * fix cpu_ram_efficient_loading to meta devices from rank 0 to prevent CPU RAM oom * improve handling of broadcasting fsdp2 state dict * support for openai chat template with thinking key as the reasoning trace * address PR feedback * refactor to remove dependency on PartialState for parallelism config * bump accelerate, gptoss fixes * limit meta fixes to fsdp2 for now * fixes for gpt oss * fixup examples, don't use cpu-ram-efficient-loading for now * remove problematic barrier * patch parallelism config * reorder comparison * device mesh fixes * make pure CP work * lint	2025-08-07 21:22:15 -04:00
NanoCode012	ca796fb56e	feat(doc): update gpt-oss readme (#3029) [skip ci] ... * feat(doc): update gpt-oss readme * fix: caps * feat: add toolcalling section * feat: add example tool dataset to docs * chore: update	2025-08-07 09:26:42 -04:00
Wing Lian	ba3dba3e4f	add kernels for gpt oss models (#3020) ... * add kernels for gpt oss models * add support for gpt-oss * typo incorrect package * fix: layout for configs and added wandb/epochs * add gptoss example w offload and set moe leaf for z3 * add support for Mxfp4Config from yaml * update yaml to use official model * fix lora and don't allow triton to go above 3.3.1 * fix lr and tweak vram use * fix range for triton since pinned wasn't compatible with toch 2.6.0 * update cce with gpt oss patches --------- Co-authored-by: NanoCode012 <nano@axolotl.ai>	2025-08-06 09:47:55 -04:00