Wing Lian
|
42410c783c
|
more fixes
|
2023-05-14 09:16:41 -04:00 |
|
Wing Lian
|
aef00b6c13
|
fix torch_dtype for model load
|
2023-05-14 08:44:22 -04:00 |
|
Wing Lian
|
aa3c3f97ae
|
optimize dataloading to use cache, fix model token embedding sizes
|
2023-05-12 13:53:27 -04:00 |
|
Wing Lian
|
2bc1a5bde1
|
black formatting
|
2023-05-10 16:01:08 -04:00 |
|
Wing Lian
|
7a490a4646
|
various fixes
|
2023-05-10 16:00:09 -04:00 |
|
Wing Lian
|
e2e68c3965
|
testing mpt triton
|
2023-05-09 20:57:40 -04:00 |
|
Wing Lian
|
a125693122
|
add support for trust_remote_code for mpt models
|
2023-05-08 12:07:27 -04:00 |
|
Wing Lian
|
247825bd57
|
refactor inference, warn if model is frozen
|
2023-05-07 01:54:15 -04:00 |
|
Wing Lian
|
7748f3d6da
|
fix adam bnb optimizer grouped parameters, fix peft model 8bit conversion logic, black formatting
|
2023-05-01 16:31:46 -04:00 |
|
Wing Lian
|
2255bb7f4f
|
support llama-adapter zero init attention
|
2023-05-01 10:42:21 -04:00 |
|
Wing Lian
|
ad2b48c0fa
|
fdsp config dict fix, todo list, add torchdistx support
|
2023-04-30 13:32:07 -04:00 |
|
Wing Lian
|
9190ada23a
|
8bit and deepspeed changes
|
2023-04-30 06:50:35 -04:00 |
|
Wing Lian
|
6dfdd2dec0
|
don't load models in 8bit unless they are using an adapter, also fix tokenizer load in exceptional case
|
2023-04-30 03:19:56 -04:00 |
|
Wing Lian
|
5159d00a86
|
fix sharegpt tokenization, refactor tokenization debugging
|
2023-04-30 00:23:53 -04:00 |
|
Wing Lian
|
4a17a4c9a1
|
fix dataset handling, support galactica
|
2023-04-24 10:54:45 -04:00 |
|
Wing Lian
|
097d367af6
|
tweaks to data loading, 8 bit adam, accelerate and deepspeed
|
2023-04-24 09:41:35 -04:00 |
|
Wing Lian
|
4f2584f2dc
|
shuffle and split dataset after save/load
|
2023-04-24 09:41:35 -04:00 |
|
Wing Lian
|
8d437853c8
|
fix sharegpt handling from hf, don't worry about loading llama if using earlier transformers release
|
2023-04-24 09:41:35 -04:00 |
|
Wing Lian
|
94f5e415a3
|
various bugfixes
|
2023-04-24 09:41:34 -04:00 |
|
Wing Lian
|
bb991fd870
|
fix bug when model_type not explicitly passed
|
2023-04-19 13:15:33 -04:00 |
|
Wing Lian
|
d65385912e
|
improve inference
|
2023-04-19 12:57:27 -04:00 |
|
Wing Lian
|
0a472e1e08
|
quickstart instructions for starting from runpod (#5)
|
2023-04-18 19:22:25 -04:00 |
|
Wing Lian
|
8746b701fe
|
attempt xformers hijack attention
|
2023-04-18 14:03:50 -04:00 |
|
Wing Lian
|
6045345d6b
|
WIP large refactor to make finetune script a little more manageable (#3)
|
2023-04-18 14:01:38 -04:00 |
|