Wing Lian
f544ab2bed
don't compile deepspeed or bitsandbytes from source ( #837 )
2023-11-08 19:49:55 -05:00
Wing Lian
8b79ff0e94
fix eval_steps to be a sane default ( #797 )
...
* fix eval_steps to be a sane default
* update docs for fractional eval_steps
2023-10-27 22:36:30 -04:00
Wing Lian
2d8def68dc
simplify by removing duplicate base_model_config ( #772 )
2023-10-23 01:42:38 -04:00
Wing Lian
e50a64e85e
prepared dataset caching, other misc fixes ( #665 )
...
* prepared dataset caching, other misc fixes
* also don't load from disk cache unless explicit
2023-10-02 21:07:24 -04:00
NanoCode012
19a600a8b8
Feat: Add support for upstream FA2 ( #626 )
...
* Feat: Add support for upstream FA2
* chore: add is_falcon_derived_model: true to examples
* chore: add config to readme for documentation
* feat: add extra model types
* fix: remove old falcon flash patch
* chore: pin transformers and accelerate
2023-09-26 09:53:28 -04:00
Morgan McGuire
7019509daa
Add wandb_entity to wandb options, update example configs, update README ( #361 )
...
* Update wandb_entity and add wandb descriptions
* add wandb to config section
* remove trailing whitespace for pre-commit hook
* remove trailing whitespace for pre-commit hook
---------
Co-authored-by: Morgan McGuire <morganmcguire@Morgans-MacBook-Pro.local >
Co-authored-by: Wing Lian <wing.lian@gmail.com >
2023-08-12 12:17:11 -04:00
Wing Lian
9a58e99e81
config fixes
2023-06-12 01:52:58 -04:00
Utensil
79a8f52181
Trim trailing whitespace
2023-06-08 23:48:57 +08:00
Utensil
a52f4816b0
Default wandb_project to empty as suggested
...
Co-authored-by: NanoCode012 <kevinvong@rocketmail.com >
2023-06-08 23:04:19 +08:00
Utensil
c9c050316f
Default micro_batch_size to 1 for a safer start
2023-06-03 17:26:33 +08:00
Utensil
ca11ae9689
Add comments/alternatives for falcon-qlora configs
2023-06-03 15:04:02 +08:00
Utensil
fb3d40f197
falcon + qlora + xformer mbs 40 gas 2 on A6000
2023-06-01 18:29:20 +08:00
Utensil
72bf8aafb6
Create config-7b-qlora.yml
2023-06-01 00:00:37 +08:00
NanoCode012
392dfd9b07
Lint and format
2023-05-31 02:53:22 +09:00
Wing Lian
157420df13
sane starter defaults and add lora
2023-05-27 09:33:14 -04:00
Wing Lian
d5f944ce2a
add example for falcoln support
2023-05-27 09:16:43 -04:00
