Wing Lian ba3dba3e4f add kernels for gpt oss models (#3020) ...

* add kernels for gpt oss models

* add support for gpt-oss

* typo incorrect package

* fix: layout for configs and added wandb/epochs

* add gptoss example w offload and set moe leaf for z3

* add support for Mxfp4Config from yaml

* update yaml to use official model

* fix lora and don't allow triton to go above 3.3.1

* fix lr and tweak vram use

* fix range for triton since pinned wasn't compatible with toch 2.6.0

* update cce with gpt oss patches

---------

Co-authored-by: NanoCode012 <nano@axolotl.ai>

2025-08-06 09:47:55 -04:00

..

alst

Distributed/ND-Parallel (#2977)

2025-07-31 15:25:02 -04:00

archived

drop old patches and code that are no longer needed (#3007) [skip ci]

2025-08-06 08:02:39 -04:00

cloud

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

cohere

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

colab-notebooks

add kernels for gpt oss models (#3020)

2025-08-06 09:47:55 -04:00

deepcogito

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

deepseek-v2

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

devstral

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

falcon-h1

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

gemma2

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

gemma3

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

gemma3n

Feat: Add voxtral, magistral small 1.1, and misc gemma3n fixes (#2979)

2025-07-30 15:57:05 +07:00

glm4

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

gpt-oss

add kernels for gpt oss models (#3020)

2025-08-06 09:47:55 -04:00

jamba

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

lfm2

make pad_to_sequence_len default to the same value as sample_packing (#2941) [skip ci]

2025-07-21 11:40:56 -04:00

llama-2

drop old patches and code that are no longer needed (#3007) [skip ci]

2025-08-06 08:02:39 -04:00

llama-3

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

llama-3-vision

fix: revert changing default optimizer to muon (#2965) [skip ci]

2025-07-22 10:00:30 -04:00

llama-4

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

llava

fix: revert changing default optimizer to muon (#2965) [skip ci]

2025-07-22 10:00:30 -04:00

magistral

Feat: Add voxtral, magistral small 1.1, and misc gemma3n fixes (#2979)

2025-07-30 15:57:05 +07:00

mamba

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

mistral

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

orpheus

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

phi

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

pixtral

fix: revert changing default optimizer to muon (#2965) [skip ci]

2025-07-22 10:00:30 -04:00

qwen2

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

qwen2_5-vl

checkpoint model on first step callback (#2906)

2025-07-15 15:00:48 -04:00

qwen2-vl

checkpoint model on first step callback (#2906)

2025-07-15 15:00:48 -04:00

qwen3

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]

2025-07-30 06:44:06 -04:00

voxtral

Feat: Add voxtral, magistral small 1.1, and misc gemma3n fixes (#2979)

2025-07-30 15:57:05 +07:00