tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Files
1d2aa1e4670012c3a4dea62b883abc489629790a
axolotl/examples
History
Wing Lian f7ea140838 TiledMLP support for FSDP2 (#2950) 
* make TiledMLP work with FSDP

* cleanup/gc at start of train to prevent large VRAM spike

* chore: lint

* generic function for non-deepspeed training

* unify patch to fix imports

* update readme for ALST and add examples

* make deepspeed attribute on params check more robust

* update with new info from PR review
2025-07-25 07:15:03 -04:00
..
alst
TiledMLP support for FSDP2 (#2950)
2025-07-25 07:15:03 -04:00
archived
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
cloud
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
cohere
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
colab-notebooks
upstream fixes in cce for dora and tensor paralel support (#2960) [skip ci]
2025-07-21 11:41:53 -04:00
deepcogito
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
deepseek-v2
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
devstral
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
falcon-h1
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
gemma2
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
gemma3
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
gemma3n
fix: revert changing default optimizer to muon (#2965) [skip ci]
2025-07-22 10:00:30 -04:00
glm4
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
jamba
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
lfm2
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
llama-2
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
llama-3
basic torchao fp8 mixed precision training (#2926)
2025-07-22 16:27:47 -04:00
llama-3-vision
fix: revert changing default optimizer to muon (#2965) [skip ci]
2025-07-22 10:00:30 -04:00
llama-4
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
llava
fix: revert changing default optimizer to muon (#2965) [skip ci]
2025-07-22 10:00:30 -04:00
magistral
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
mamba
checkpoint model on first step callback (#2906)
2025-07-15 15:00:48 -04:00
mistral
Feat: add gemma3n support (#2852)
2025-07-22 16:52:15 +07:00
orpheus
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
phi
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
pixtral
fix: revert changing default optimizer to muon (#2965) [skip ci]
2025-07-22 10:00:30 -04:00
qwen2
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00
qwen2_5-vl
checkpoint model on first step callback (#2906)
2025-07-15 15:00:48 -04:00
qwen2-vl
checkpoint model on first step callback (#2906)
2025-07-15 15:00:48 -04:00
qwen3
make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]
2025-07-21 11:40:56 -04:00