tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

add 120b and deepspeed zero3 examples (#3035) [skip ci]

Browse Source 
* add 120b and deepspeed zero3 examples

* add a bit of flavor and cleanup gpt oss readme

* fix: remove expert vram usage

* fix: remove redundant EOS token from eot_tokens

* feat: add 120B to docs

---------

Co-authored-by: NanoCode012 <nano@axolotl.ai>
This commit is contained in:
Wing Lian
2025-08-08 08:04:56 -04:00
committed by GitHub
parent eb2c87b525
commit 50f2b94d50
6 changed files with 141 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/gpt-oss/gpt-oss-20b-fft-fsdp2.yaml
View File

@@ -53,7 +53,6 @@ warmup_ratio: 0.03
special_tokens:
eot_tokens:
- "<|end|>"
- "<|return|>"
fsdp_version: 2
fsdp_config: