Wing Lian e4063d60a7 bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769) ...

* bump transformers and set roundup_power2_divisions for more VRAM improvements

* support for low bit optimizers from torch ao

* fix check for alternate optimizers and use nous models on hf for llama3

* add missing check for ao_adamw_fp8

* fix check when using custom optimizers w adamw

2024-07-19 00:47:07 -04:00

..

dataset-formats

improve Pre-Tokenized Dataset docs (#1684) [skip ci]

2024-06-26 13:13:21 -07:00

images

support for true batches with multipack (#1230)

2024-02-01 10:18:42 -05:00

.gitignore

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

batch_vs_grad.qmd

Feat: update doc (#1475) [skip ci]

2024-04-04 13:43:40 +09:00

config.qmd

bump trl and accelerate for latest releases (#1730)

2024-07-10 11:15:44 -04:00

dataset_preprocessing.qmd

add docs around pre-processing (#1529)

2024-04-16 19:45:46 -04:00

debugging.qmd

github urls (#1734)

2024-07-11 09:19:29 -04:00

faq.qmd

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

fsdp_qlora.qmd

github urls (#1734)

2024-07-11 09:19:29 -04:00

input_output.qmd

github urls (#1734)

2024-07-11 09:19:29 -04:00

mac.qmd

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

multi-node.qmd

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

multipack.qmd

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

nccl.qmd

Bootstrap Hosted Axolotl Docs w/Quarto (#1429)

2024-03-21 22:28:36 -07:00

rlhf.qmd

Add ORPO example and e2e test (#1572)

2024-04-27 12:07:06 -04:00

torchao.qmd

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769)

2024-07-19 00:47:07 -04:00

unsloth.qmd

Unsloth rope (#1767)

2024-07-18 14:54:41 -04:00