Files

VED 1eaf4d7418 add: support mxfp4 axo (#3375 )

* mxfp4 axo

* import lint

* test for qat mxfp4

* config for mxfp4

* add qat:

* pass base config

* MXFakeQuantizeConfig

* lint

* tune config so it fits in 32GB VRAM

---------

Co-authored-by: Wing Lian <wing@axolotl.ai>

2026-03-05 13:40:45 -05:00

diffusion

text diffusion training plugin (#3067 )

2025-09-10 20:27:00 -04:00

3b-fp8-fsdp2.yaml

fix check for fp8 capability (#3324 )

2025-12-22 13:58:25 -05:00

3b-qat-fsdp2.yaml

Migrate QAT API; fix axolotl quantize for QAT-ed models; add NVFP4 (#3107 )

2025-09-12 10:55:50 +01:00

3b-qat-mxfp4.yaml

add: support mxfp4 axo (#3375 )

2026-03-05 13:40:45 -05:00

3b-qat-nvfp4.yaml

qat doc updates (#3162 ) [skip-ci]

2025-09-17 10:38:15 +01:00

fft-8b-liger-fsdp.yaml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

fft-8b.yaml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

instruct-dpo-lora-8b.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

instruct-lora-8b.yml

fix: unify default for conversations_field [skip-e2e] (#3070 )

2025-09-23 21:22:15 +07:00

lora-1b-deduplicate-dpo.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

lora-1b-deduplicate-sft.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

lora-1b-kernels.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

lora-1b-ray.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

lora-1b-sample-packing-sequentially.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

lora-1b.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

lora-8b.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

opentelemetry-qlora.yml

Feat/opentelemetry (#3215 )

2025-10-22 19:16:55 -07:00

qlora-1b-gdpo.yaml

feat: Add GDPO Support (#3353 )

2026-01-21 17:22:45 -05:00

qlora-1b-kto.yaml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

qlora-1b.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

qlora-fsdp-70b.yaml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

qlora-fsdp-405b.yaml

transformers v5 upgrade (#3272 )

2026-01-27 17:08:24 -05:00

qlora.yml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

README.md

llama-3 examples (#1537 )

2024-04-18 14:28:03 -04:00

sparse-finetuning.yaml

use warmup_ratio as a better default than warmup steps since it's data dependent (#2897 ) [skip ci]

2025-07-30 06:44:06 -04:00

README.md

Llama-3

https://llama.meta.com/llama3/

8B Base Model

Full Fine Tune
- Single GPU @ 48GB VRAM
LoRA
- Single GPU @ 11GB VRAM

70B Base Model

QLORA+FSDP
- Dual GPU @ 21GB VRAM