Wing Lian
2d8def68dc
simplify by removing duplicate base_model_config ( #772 )
2023-10-23 01:42:38 -04:00
Wing Lian
21cf09b608
remove lora fused packing test ( #758 )
2023-10-21 22:59:35 -04:00
Casper
15d3a654bf
Implement fused modules ( #747 )
...
* MLP: Memory saving
* Remove RMSNorm restrictions
* Map packed weights to original
* FusedAttention module
* Simplify code
* Move fused modules
* Fix critical typo
* Split inplace
* Add FFT config
* Add validation of fused arguments
* Add fused arguments to config
* Update docs
* Fix validation logic
* Add fused modules to flash attn
* Only fuse during training
* Remove timing
* Formatting
* Formatting
* Formatting
* chore: lint
* chore: lint
* add e2e tests for fused llama
* no lora for tests
---------
Co-authored-by: Wing Lian <wing.lian@gmail.com >
2023-10-21 16:08:25 -04:00
NanoCode012
697c50d408
Feat: Allow usage of native Mistral FA when no sample_packing ( #669 )
...
* Allow usage of native Mistral FA when no sample_packing
* fix: do not apply custom patch when sample_pack off
* chore: lint
* chore: pin transformer to v4.35.0.dev0
* fix: split sample_packing to separate test
2023-10-04 20:40:47 +09:00
Wing Lian
5b0bc48fbc
add mistral e2e tests ( #649 )
...
* mistral e2e tests
* make sure to enable flash attention for the e2e tests
* use latest transformers full sha
* uninstall first
2023-09-29 00:22:40 -04:00
Wing Lian
03e59077a0
misc fixes to add gptq tests ( #621 )
...
* misc fixes to add gptq tests
* set bf16 needed for fa2
2023-09-21 21:52:12 -04:00
Wing Lian
12a2dbbc2c
Support Sample packing for phi arch ( #586 )
...
* phi sequence packing
* sample packing fixes
* fix linting
* fix inference and phi e2e tests
* update phi example now that sample packing works
* wandb import keeps getting moved around
2023-09-15 15:46:54 -04:00
Wing Lian
24146733db
E2e device cuda ( #575 )
...
* use torch.cuda.current_device() instead of local_rank
* ignore NVML errors for gpu stats
* llama lora packing e2e tests
2023-09-14 22:49:27 -04:00
Wing Lian
9218ebecd2
e2e testing ( #574 )
2023-09-14 21:56:11 -04:00
