Wing Lian 3ebf22464b qlora-fsdp ram efficient loading with hf trainer (#1791) ...

* fix 405b with lower cpu ram requirements

* make sure to use doouble quant and only skip output embeddings

* set model attributes

* more fixes for sharded fsdp loading

* update the base model in example to use pre-quantized nf4-bf16 weights

* upstream fixes  for qlora+fsdp

2024-07-30 19:21:38 -04:00

..

cerebras

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

code-llama

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

colab-notebooks

typo (#1685) [skip ci]

2024-07-12 21:24:01 -04:00

dbrx

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

falcon

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

gemma

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

gemma2

support for gemma2 w sample packing (#1718)

2024-06-29 01:38:55 -04:00

gptj

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

jamba

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

jeopardy-bot

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

llama-2

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

llama-3

qlora-fsdp ram efficient loading with hf trainer (#1791)

2024-07-30 19:21:38 -04:00

mamba

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

mistral

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

mpt-7b

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

openllama-3b

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

phi

Create phi3-ft-fsdp.yml (#1580)

2024-06-04 16:20:25 -04:00

pythia

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

pythia-12b

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

qwen

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

qwen2

add qwen2-72b fsdp example (#1696)

2024-06-07 16:38:29 -04:00

redpajama

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

replit-3b

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

stablelm-2

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

starcoder2

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

tiny-llama

Update tiny-llama qlora.yml addressing eval packing error (#1638)

2024-05-22 08:34:06 -04:00

xgen-7b

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00

yi-34B-chat

update outputs path so that we can mount workspace to /workspace/data (#1623)

2024-05-15 12:44:13 -04:00