Files

Wing Lian e4063d60a7 bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 )

* bump transformers and set roundup_power2_divisions for more VRAM improvements

* support for low bit optimizers from torch ao

* fix check for alternate optimizers and use nous models on hf for llama3

* add missing check for ao_adamw_fp8

* fix check when using custom optimizers w adamw

2024-07-19 00:47:07 -04:00

fft-8b.yaml

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 )

2024-07-19 00:47:07 -04:00

instruct-lora-8b.yml

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 )

2024-07-19 00:47:07 -04:00

lora-8b.yml

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers (#1769 )

2024-07-19 00:47:07 -04:00

qlora-fsdp-70b.yaml

update outputs path so that we can mount workspace to /workspace/data (#1623 )