tocmo0nlord
0 Followers · 0 Following
  • Joined on 2026-02-11
Repositories
11
 Projects Packages Public Activity Starred Repositories
tocmo0nlord pushed to fsdp2_fp32 at tocmo0nlord/axolotl 2026-04-26 15:07:17 +00:00
1d0562dedd adding fp32 support
tocmo0nlord created branch fused-mlp-ez in tocmo0nlord/axolotl 2026-04-26 15:07:17 +00:00
tocmo0nlord created branch fsdp-fix in tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
tocmo0nlord pushed to fsdp-fix at tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
744f7082f5 fix for fsdp for models that aren't qwen2 or jamba
tocmo0nlord created branch fsdp-qdora in tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
tocmo0nlord pushed to fsdp-qdora at tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
7a7c56f018 fixes to support fsdp-qdora
tocmo0nlord created branch fsdp2 in tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
tocmo0nlord pushed to fsdp2 at tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
c7f1c191a3 additional validation for fsdp2, bump dep versions
1a5d445413 make sure to patch all the loaded models
7e410ab480 more fixes to flex for fsdp2
b5a51c378b okay, actually use fdsp2...
c902f4222d make sure both flex and flash attn work with fsdp2, skip fix untrained tokens
Compare 10 commits »
tocmo0nlord created branch fsdp2_fp32 in tocmo0nlord/axolotl 2026-04-26 15:07:16 +00:00
tocmo0nlord created branch fp8 in tocmo0nlord/axolotl 2026-04-26 15:07:15 +00:00
tocmo0nlord pushed to fp8 at tocmo0nlord/axolotl 2026-04-26 15:07:15 +00:00
8836986a92 support for fp8
tocmo0nlord created branch fsdp-defaults in tocmo0nlord/axolotl 2026-04-26 15:07:15 +00:00
tocmo0nlord pushed to fsdp-defaults at tocmo0nlord/axolotl 2026-04-26 15:07:15 +00:00
53ce90d21e add sync_model_states parameter to fix resume from checkpoint with fsdp
tocmo0nlord created branch fsdp-fft in tocmo0nlord/axolotl 2026-04-26 15:07:15 +00:00
tocmo0nlord pushed to fsdp-fft at tocmo0nlord/axolotl 2026-04-26 15:07:15 +00:00
2b890ead05 fsdp fft loading on meta device
34de5b3bd5 extras for the various flash attn subdirs and build those in the base module as it is a slow step
a1d168d314 break out the additional llama patches from the flash attn w multipack patch
Compare 2 commits »
tocmo0nlord created branch flex_patching_update in tocmo0nlord/axolotl 2026-04-26 15:07:14 +00:00
tocmo0nlord pushed to flex_patching_update at tocmo0nlord/axolotl 2026-04-26 15:07:14 +00:00
deb01959d2 raising value error
76ae4ae238 Merge branch 'main' into flex_patching_update
Compare 2 commits »
tocmo0nlord created branch flx_attn_support in tocmo0nlord/axolotl 2026-04-26 15:07:14 +00:00
tocmo0nlord pushed to flx_attn_support at tocmo0nlord/axolotl 2026-04-26 15:07:14 +00:00
328bb0466b Merge branch 'main' into flx_attn_support
e792b54bab remove unnecessary components
Compare 2 commits »