Wing Lian
|
c2a0792680
|
swap batch size for gradient accumulation steps to decouple from num gpu
|
2023-05-31 09:38:12 -04:00 |
|
Viktorius Suwandi
|
d77d736631
|
Update wandb_log_model on llama_7B_alpaca.yml
|
2023-05-29 15:43:01 +07:00 |
|
Wing Lian
|
87e073d0de
|
fix lora target module, require explicit flash attention, fix min logging steps, don't use adam8bit for int4, hash prepared datasets, support hf hub datasets
|
2023-04-17 18:01:12 -04:00 |
|
Wing Lian
|
77fca25f1b
|
4bit quantized support (wip)
|
2023-04-17 11:37:39 -04:00 |
|
Wing Lian
|
d1aed4c8e5
|
deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches
|
2023-04-16 06:59:47 -04:00 |
|
Wing Lian
|
d060c803ce
|
add llama 7b config and fiz lora_fan_in_fan_out for llama (copy pasta bug)
|
2023-04-15 14:26:52 -04:00 |
|