tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

add 120b and deepspeed zero3 examples (#3035) [skip ci]

Browse Source 
* add 120b and deepspeed zero3 examples

* add a bit of flavor and cleanup gpt oss readme

* fix: remove expert vram usage

* fix: remove redundant EOS token from eot_tokens

* feat: add 120B to docs

---------

Co-authored-by: NanoCode012 <nano@axolotl.ai>
This commit is contained in:
Wing Lian
2025-08-08 08:04:56 -04:00
committed by GitHub
parent eb2c87b525
commit 50f2b94d50
6 changed files with 141 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
examples/gpt-oss/gpt-oss-20b-fft-fsdp2-offload.yaml
View File

@@ -54,7 +54,6 @@ warmup_ratio: 0.03
special_tokens:
eot_tokens:
- "<|end|>"
- "<|return|>"
fsdp_version: 2
fsdp_config:
@@ -63,4 +62,7 @@ fsdp_config:
auto_wrap_policy: TRANSFORMER_BASED_WRAP
transformer_layer_cls_to_wrap: GptOssDecoderLayer
reshard_after_forward: true
# cpu_ram_efficient_loading: true
# cpu_ram_efficient_loading: true
# cpu_ram_efficient_loading cannot be used with MXFP4 model quantization.
# It can only be used with a dequantized model like `axolotl-ai-co/gpt-oss-120b-dequantized`