tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Files
v0.13.2
axolotl/examples/llama-3
History
VED d0d26d5064 feat: Add GDPO Support (#3353) 
* gdpo support - test left

* lint

* fixxes for vllm serv

* test advantages

* docss

* lint

* lint =

* gdpo simple + lint

* lint nit

* example

* lint

* trl 0.27.0

* blocklist

* test assert rmv

* add validation check for GDPO + sum_then_normalize

---------

Co-authored-by: Wing Lian <wing@axolotl.ai>
2026-01-21 17:22:45 -05:00
..
diffusion
text diffusion training plugin (#3067)
2025-09-10 20:27:00 -04:00
3b-fp8-fsdp2.yaml
fix check for fp8 capability (#3324)
2025-12-22 13:58:25 -05:00
3b-qat-fsdp2.yaml
Migrate QAT API; fix axolotl quantize for QAT-ed models; add NVFP4 (#3107)
2025-09-12 10:55:50 +01:00
3b-qat-nvfp4.yaml
qat doc updates (#3162) [skip-ci]
2025-09-17 10:38:15 +01:00
fft-8b-liger-fsdp.yaml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
fft-8b.yaml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
instruct-dpo-lora-8b.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
instruct-lora-8b.yml
fix: unify default for conversations_field [skip-e2e] (#3070)
2025-09-23 21:22:15 +07:00
lora-1b-deduplicate-dpo.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
lora-1b-deduplicate-sft.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
lora-1b-kernels.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
lora-1b-ray.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
lora-1b-sample-packing-sequentially.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
lora-1b.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
lora-8b.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
opentelemetry-qlora.yml
Feat/opentelemetry (#3215)
2025-10-22 19:16:55 -07:00
qlora-1b-gdpo.yaml
feat: Add GDPO Support (#3353)
2026-01-21 17:22:45 -05:00
qlora-1b-kto.yaml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
qlora-1b.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
qlora-fsdp-70b.yaml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
qlora-fsdp-405b.yaml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
qlora.yml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00
README.md
llama-3 examples (#1537)
2024-04-18 14:28:03 -04:00
sparse-finetuning.yaml
use warmup_ratio as a better default than warmup steps since it's data dependent (#2897) [skip ci]
2025-07-30 06:44:06 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

8B Base Model

70B Base Model

Reference in New Issue View Git Blame Copy Permalink