tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
873 Commits 319 Branches 32 Tags
3355706e22d95ddca7dc242fe429f06d4ce89526
Commit Graph

50 Commits

Author SHA1 Message Date
Wing Lian
3355706e22 Add support for GPTQ using native transformers/peft (#468) 
* auto gptq support

* more tweaks and add yml

* remove old gptq docker

* don't need explicit peft install for tests

* fix setup.py to use extra index url

install torch for tests
fix cuda version for autogptq index
set torch in requirements so that it installs properly
move gptq install around to work with github cicd

* gptq doesn't play well with sample packing

* address pr feedback

* remove torch install for now

* set quantization_config from model config

* Fix the implementation for getting quant config from model config
 2023-09-05 12:43:22 -04:00
Wing Lian
96deb6bd67 recast loralayer, norm, lmhead + embed token weights per original qlora (#393) 
* recast loralayer, norm, lmhead + embed token weights per original qlora

* try again for the fix

* refactor torch dtype picking

* linter fixes

* missing import for LoraLayer

* fix install for tests now that peft is involved
 2023-08-21 18:41:12 -04:00
mhenrichsen
cf6654769a flash attn pip install (#426) 
* flash attn pip

* add packaging

* add packaging to apt get

* install flash attn in dockerfile

* remove unused whls

* add wheel

* clean up pr

fix packaging requirement for ci
upgrade pip for ci
skip build isolation for requiremnents to get flash-attn working
install flash-attn seperately

* install wheel for ci

* no flash-attn for basic cicd

* install flash-attn as pip extras

---------

Co-authored-by: Ubuntu <mgh@mgh-vm.wsyvwcia0jxedeyrchqg425tpb.ax.internal.cloudapp.net>
Co-authored-by: mhenrichsen <some_email@hey.com>
Co-authored-by: Mads Henrichsen <mads@BrbartiendeMads.lan>
Co-authored-by: Wing Lian <wing.lian@gmail.com>
 2023-08-18 19:00:27 -04:00
Wing Lian
d3d6fd6ae6 just resort to tags ans use main-latest (#424) 2023-08-16 00:39:57 -04:00
Wing Lian
5f80b3560b use inputs for image rather than outputs for docker metadata (#420) 2023-08-15 18:26:59 -04:00
Wing Lian
7af816699e tag with latest as well for axolotl-runpod (#418) 
* tag with latest as well for axolotl-runpod

* no dev branch for now
 2023-08-15 15:30:41 -04:00
Wing Lian
918f1b0dfb revert previous change and build ax images w docker on gpu (#371) 2023-08-12 20:23:00 -04:00
Wing Lian
c3fde36ada attempt to run non-base docker builds on regular cpu hosts (#369) 2023-08-12 19:07:38 -04:00
Wing Lian
2c37bf6c21 Prune cuda117 (#327) 
* drop cuda117/torch 1.13.1 from support, pin flash attention to v2.0.1, rm torchvision/torchaudio install

* gptq base build not needed. add sm 9.0 support
 2023-07-26 16:27:49 -04:00
Wing Lian
ff7f18d1ed disable gh cache for first step of docker builds too 2023-07-22 11:46:37 -04:00
Wing Lian
cf62cfd661 add runpod envs to .bashrc, fix bnb env (#316) 
* hopper support for base dockerfile, add runpod envs to .bashrc

* set BNB_CUDA_VERSION env for latest bnb

* don't support hopper yet w 118
 2023-07-22 10:09:38 -04:00
Wing Lian
c5df969262 don't use the gha cache w docker 2023-07-22 08:46:21 -04:00
Wing Lian
c58034d48c use pytorch 2.0.1 2023-07-20 00:47:13 -04:00
Wing Lian
a10da1caff 11.7.0 nvidia/cuda docker images are deprecated, move to 11.7.1
Some checks failed
ci-cd-base / build-base (<nil>, 117, 11.7.1, 3.9, 1.13.1) (push) Has been cancelled
Details
ci-cd-base / build-base (<nil>, 118, 11.8.0, 3.10, 2.0.0) (push) Has been cancelled
Details
ci-cd-base / build-base (<nil>, 118, 11.8.0, 3.9, 2.0.0) (push) Has been cancelled
Details
ci-cd-base / build-base (gptq, 118, 11.8.0, 3.9, 2.0.0) (push) Has been cancelled
Details
pre-commit / pre-commit (push) Has been cancelled
Details
PyTest / test (3.10) (push) Has been cancelled
Details
PyTest / test (3.9) (push) Has been cancelled
Details
2023-07-01 00:29:07 -04:00
Wing Lian
d35278aaf1 don't fail fast 2023-06-15 16:01:27 -04:00
Wing Lian
e0011fdf55 Fix base builder, missing tags 2023-05-31 09:52:03 -04:00
Wing Lian
e3d03745ba add py310 support from base image 2023-05-31 09:07:28 -04:00
Wing Lian
c5b0af1a7e define python version (3.10) explicitly as string in yaml 2023-05-30 22:23:35 -04:00
Wing Lian
c43c5c84ff py310, fix cuda arg in deepspeed 2023-05-30 18:02:34 -04:00
Wing Lian
bbc5bc5791 Merge pull request #108 from OpenAccess-AI-Collective/docker-gptq 
default to qlora support, make gptq specific image
 2023-05-30 15:07:04 -04:00
NanoCode012
36596adaf7 Add pre-commit: black+flake8+pylint 2023-05-31 02:53:22 +09:00
Wing Lian
48612f8376 cleanup from pr feedback 2023-05-30 09:56:30 -04:00
Wing Lian
6ef96f569b default to qlora support, make gptq specific image 2023-05-29 20:34:41 -04:00
Wing Lian
e43bcc6c4f move CUDA_VERSION_BNB arg inside of stage build scope 2023-05-29 13:30:15 -04:00
Wing Lian
21f17cca69 bnb fixes 2023-05-29 00:06:35 -04:00
Wing Lian
658ed86cb5 add py310 to the test matrix 2023-05-28 14:25:57 -04:00
NanoCode012
dd83a20c27 Update test to run on PR 2023-05-28 23:30:17 +09:00
Wing Lian
af3aacbe16 Merge pull request #93 from OpenAccess-AI-Collective/dev-base 
cuda properly compiled bitsandbytes for qlora support
 2023-05-27 19:40:29 -04:00
NanoCode012
2b8c28bab8 Update actions version 2023-05-28 01:51:10 +09:00
Wing Lian
312b8d51d6 update docker to compile latest bnb to properly support qlora 2023-05-27 12:36:53 -04:00
Wing Lian
a653392287 use requirements file for tests 2023-05-27 12:17:46 -04:00
Wing Lian
72b6ca0d9f cache pip 2023-05-27 12:16:54 -04:00
Wing Lian
7f53fd2ab6 alright, just local install it 2023-05-27 12:16:06 -04:00
Wing Lian
c29d33352c move python path to same step as tests 2023-05-27 12:06:23 -04:00
Wing Lian
403af0b1d7 fix path and streamline pip installs 2023-05-27 11:58:37 -04:00
Wing Lian
d199d6c261 automated testing in github actions 2023-05-27 11:51:01 -04:00
Wing Lian
e3df3a9f5d cuda/pytorch matrix builds 2023-05-22 12:14:21 -04:00
Wing Lian
f950a881e1 cuda, pytorch matrix for base builds 2023-05-22 12:12:08 -04:00
Wing Lian
79deb35c68 setup runpod images 
use github.ref_name
 2023-05-08 10:48:32 -04:00
Wing Lian
fae36c7111 blah, wrong base tag 2023-05-07 17:54:26 -04:00
Wing Lian
9cd5d3fcfc build on self hosted GPU runners 2023-05-07 17:25:31 -04:00
Wing Lian
990bec63e6 docker layer caching, build w axolotl from base build 2023-05-07 17:16:05 -04:00
Wing Lian
2734e3f1a2 build base separately 
fix arg order for image
fix dockerfile var excaping
move args around
 2023-05-07 10:56:12 -04:00
Wing Lian
14ebd2e007 build base too 2023-05-07 09:48:41 -04:00
Wing Lian
4a79dabff0 fix push to docker hub 2023-05-07 08:52:49 -04:00
Wing Lian
47ad3890bc fix whitespace and instruction on inference 2023-05-07 08:28:15 -04:00
Wing Lian
76b24bca2e push to docker hub 
set docker image name
 2023-05-07 08:06:50 -04:00
Wing Lian
97cf77891e run this on self hosted runner for now 
fix typo
fixes to docker build
need pip wheel
don't duplicate pip install
 2023-05-07 07:21:25 -04:00
Wing Lian
e2599edab9 runs on larger git runner? 2023-05-07 04:12:47 -04:00
Wing Lian
2634689774 build dockerfile in gha 2023-05-07 02:58:21 -04:00