tocmo0nlord
0 Followers · 0 Following
  • Joined on 2026-02-11
Repositories
11
 Projects Packages Public Activity Starred Repositories
tocmo0nlord created branch shampoo in tocmo0nlord/axolotl 2026-04-26 15:07:57 +00:00
tocmo0nlord pushed to shampoo at tocmo0nlord/axolotl 2026-04-26 15:07:57 +00:00
17330c05a3 shampoo checkpoint save workaround
992ea517b7 setup precision config for bf16
beaee36191 ddp shampoo
69a29382e1 fix casting of optim args
84dad0bd12 ensure epsilon is cast to float
Compare 9 commits »
tocmo0nlord created branch shampoo-low_bit in tocmo0nlord/axolotl 2026-04-26 15:07:57 +00:00
tocmo0nlord created branch seq-parallel-ring in tocmo0nlord/axolotl 2026-04-26 15:07:56 +00:00
tocmo0nlord pushed to seq-parallel-ring at tocmo0nlord/axolotl 2026-04-26 15:07:56 +00:00
ee489d16bf wip
d88e071120 sequence parallelism
Compare 2 commits »
tocmo0nlord created branch sequence-parallelism in tocmo0nlord/axolotl 2026-04-26 15:07:56 +00:00
tocmo0nlord pushed to sequence-parallelism at tocmo0nlord/axolotl 2026-04-26 15:07:56 +00:00
4ac65462f0 precommit
ce35b2a95f precommit
ab3b36339a fix tests
22cfa42961 small updates
0b2c2ed68c refactors, SP mixin
Compare 10 commits »
tocmo0nlord created branch scattermoe-lora-optim-dtypestest in tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
tocmo0nlord pushed to scattermoe-lora-optim-dtypestest at tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
42922f8f8b register pressure estimation and pruning for h200/b200
7041592ca7 fix casting for H200 and B200
fec0c3a99e chore: lint
31d8d068bb handle base+lora split kernel for older moe models
66fea258c7 add correctness unit tests and benchmarks for scattermoe + lora
Compare 8 commits »
tocmo0nlord created branch scattermoe-nanotron in tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
tocmo0nlord pushed to scattermoe-nanotron at tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
936149380f support nemotron for scattermoe-lora
tocmo0nlord created branch sdpa-cp in tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
tocmo0nlord pushed to sdpa-cp at tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
cbcc795bb3 commenting out unused
e34b6f4dfe temp: trying another approach
f8f87321bd progress
7a88de4fa8 finish basic impl; change naming from SP -> CP to match torch
aced809989 progress (messy :O)
Compare 7 commits »
tocmo0nlord created branch sdpa-multipack in tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
tocmo0nlord pushed to sdpa-multipack at tocmo0nlord/axolotl 2026-04-26 15:07:55 +00:00
1a538be9c2 add a prelim test for expading the 4d mask
tocmo0nlord created branch scatter_moe in tocmo0nlord/axolotl 2026-04-26 15:07:54 +00:00
tocmo0nlord pushed to scatter_moe at tocmo0nlord/axolotl 2026-04-26 15:07:54 +00:00
10328b3429 Simplify creating parameters
5bfc470d57 Stop transformers from using all memory
04168801c9 Simplify conversion + more debug
d43a79b7bf device_map auto
884d81331e Initialize ParallelExperts on device of first expert
Compare 6 commits »
tocmo0nlord created branch scatter_moe_eric in tocmo0nlord/axolotl 2026-04-26 15:07:54 +00:00
tocmo0nlord pushed to scatter_moe_eric at tocmo0nlord/axolotl 2026-04-26 15:07:54 +00:00
9c221a6761 code review feedback
301cc4c006 implement post training
Compare 2 commits »
tocmo0nlord pushed to save_only_model at tocmo0nlord/axolotl 2026-04-26 15:07:53 +00:00
3202f19f52 add save_only_model arg