tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
874 Commits 319 Branches 32 Tags
881d333b84b4e1055688d2e4d1d1262fec0f3979
Commit Graph

54 Commits

Author SHA1 Message Date
Wing Lian
3355706e22 Add support for GPTQ using native transformers/peft (#468) 
* auto gptq support

* more tweaks and add yml

* remove old gptq docker

* don't need explicit peft install for tests

* fix setup.py to use extra index url

install torch for tests
fix cuda version for autogptq index
set torch in requirements so that it installs properly
move gptq install around to work with github cicd

* gptq doesn't play well with sample packing

* address pr feedback

* remove torch install for now

* set quantization_config from model config

* Fix the implementation for getting quant config from model config
 2023-09-05 12:43:22 -04:00
Aman Gupta Karmani
e356b297cb remove --force-reinstall from Dockerfile to ensure correct pytorch version (#492) 2023-08-29 06:17:51 -07:00
mhenrichsen
cf6654769a flash attn pip install (#426) 
* flash attn pip

* add packaging

* add packaging to apt get

* install flash attn in dockerfile

* remove unused whls

* add wheel

* clean up pr

fix packaging requirement for ci
upgrade pip for ci
skip build isolation for requiremnents to get flash-attn working
install flash-attn seperately

* install wheel for ci

* no flash-attn for basic cicd

* install flash-attn as pip extras

---------

Co-authored-by: Ubuntu <mgh@mgh-vm.wsyvwcia0jxedeyrchqg425tpb.ax.internal.cloudapp.net>
Co-authored-by: mhenrichsen <some_email@hey.com>
Co-authored-by: Mads Henrichsen <mads@BrbartiendeMads.lan>
Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2023-08-18 19:00:27 -04:00
Wing Lian
ffac902c1b bump flash-attn to 2.0.4 for the base docker image (#382) 2023-08-13 17:55:04 -04:00
Wing Lian
db2a3586f3 add peft install back since it doesn't get installed by setup.py (#331) 2023-07-31 16:31:53 -04:00
Wing Lian
6c9a87c8ee pin accelerate so it works with llama2 (#330) 2023-07-30 22:20:06 -04:00
Wing Lian
2c37bf6c21 Prune cuda117 (#327) 
* drop cuda117/torch 1.13.1 from support, pin flash attention to v2.0.1, rm torchvision/torchaudio install

* gptq base build not needed. add sm 9.0 support
 2023-07-26 16:27:49 -04:00
Wing Lian
cf62cfd661 add runpod envs to .bashrc, fix bnb env (#316) 
* hopper support for base dockerfile, add runpod envs to .bashrc

* set BNB_CUDA_VERSION env for latest bnb

* don't support hopper yet w 118
 2023-07-22 10:09:38 -04:00
Wing Lian
cdf85fdbd5 pin flash attention 2 to the fix for backwards pass 2023-07-21 08:18:53 -04:00
Wing Lian
9b790d359b flash attention 2 2023-07-21 08:17:46 -04:00
Wing Lian
b06d3e3645 explicitly pin flash attention 1 to v1.0.9 2023-07-20 01:02:08 -04:00
Wing Lian
d75adb9835 misc fixes 2023-07-17 03:00:27 -04:00
Wing Lian
f162f3c7cc set transformers cache env var in docker image 2023-07-16 23:03:54 -04:00
Wing Lian
eca3531329 git fetch fix for docker 2023-07-16 22:25:05 -04:00
Wing Lian
71456955f5 pin pydantic so deepspeed isn't broken 2023-07-02 22:26:51 -04:00
Wing Lian
530809fd74 update pip install command for apex 2023-06-28 22:36:28 -04:00
Wing Lian
5cd2126439 shallow clone 2023-06-02 14:54:28 -04:00
Wing Lian
12620f3089 clone in docker 2023-06-02 14:52:50 -04:00
Wing Lian
c43c5c84ff py310, fix cuda arg in deepspeed 2023-05-30 18:02:34 -04:00
Wing Lian
bbc5bc5791 Merge pull request #108 from OpenAccess-AI-Collective/docker-gptq 
default to qlora support, make gptq specific image
 2023-05-30 15:07:04 -04:00
NanoCode012
392dfd9b07 Lint and format 2023-05-31 02:53:22 +09:00
Wing Lian
48612f8376 cleanup from pr feedback 2023-05-30 09:56:30 -04:00
Wing Lian
6ef96f569b default to qlora support, make gptq specific image 2023-05-29 20:34:41 -04:00
Wing Lian
e43bcc6c4f move CUDA_VERSION_BNB arg inside of stage build scope 2023-05-29 13:30:15 -04:00
Wing Lian
00323f0a6f fix CUDA_VERSION_BNB env var 2023-05-29 08:06:22 -04:00
Wing Lian
21f17cca69 bnb fixes 2023-05-29 00:06:35 -04:00
Wing Lian
809ccebb38 use python setup install, bdist wheel is unreliable in installing extension 2023-05-28 15:49:13 -04:00
Wing Lian
a798ba1659 ensure libbitsandbytes*.so gets included with wheel 2023-05-28 12:28:37 -04:00
Wing Lian
cf37980395 fix missing run coninuation 2023-05-27 15:28:54 -04:00
Wing Lian
312b8d51d6 update docker to compile latest bnb to properly support qlora 2023-05-27 12:36:53 -04:00
Wing Lian
c3d256271e fix wheel install glob 2023-05-26 10:37:02 -04:00
Wing Lian
1fc9b44e3d fix wheel blobs in dockerfile 2023-05-26 07:40:11 -04:00
Wing Lian
259262bf42 fix xentropy wheel name typo 2023-05-25 17:25:38 -04:00
Wing Lian
8d6a28953f fix relative path in flash-attn build: 2023-05-25 12:18:28 -04:00
Wing Lian
f5fa3d131b fix cd within flash-attn 2023-05-25 11:29:15 -04:00
Wing Lian
1d5ab84486 optionally be able to specify alpaca or chat style prompts 2023-05-20 18:16:22 -04:00
Wing Lian
fa8bd14be4 update entrypoint and force min accelerate 2023-05-18 06:25:34 -04:00
Wing Lian
709be5af81 use printf instead of echo in dockerfile for portability 2023-05-08 11:45:38 -04:00
Wing Lian
807cca81c0 fix path name to sorkspace 2023-05-08 11:20:03 -04:00
Wing Lian
79deb35c68 setup runpod images 
use github.ref_name
 2023-05-08 10:48:32 -04:00
Wing Lian
7576d85c73 fix to cd to path in docker 2023-05-08 03:43:46 -04:00
Wing Lian
3b4b476828 use existing state of repo to build, not the checkout 2023-05-08 03:29:48 -04:00
Wing Lian
b5fe063687 fix base for dockerfile 2023-05-08 03:27:10 -04:00
Wing Lian
a31746baa2 whoops, build from base image 2023-05-07 17:47:54 -04:00
Wing Lian
17345c8a4b hanging slash typo 2023-05-07 17:38:56 -04:00
Wing Lian
990bec63e6 docker layer caching, build w axolotl from base build 2023-05-07 17:16:05 -04:00
Wing Lian
0c46806ae2 typo in git repo for pip 2023-05-07 16:00:21 -04:00
Wing Lian
66fa751c18 add huggingface packages and awscli 2023-05-07 11:51:57 -04:00
Wing Lian
21b74397de fix typo and add apex 2023-05-07 11:48:47 -04:00
Wing Lian
3f11b47488 needs libaio-dev from apt 2023-05-07 11:23:43 -04:00