tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
714 Commits 319 Branches 32 Tags
db2a3586f3757123a7951dfeca374061cc384f01
Commit Graph

714 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wing Lian
db2a3586f3 add peft install back since it doesn't get installed by setup.py (#331) 2023-07-31 16:31:53 -04:00
Wing Lian
6c9a87c8ee pin accelerate so it works with llama2 (#330) 2023-07-30 22:20:06 -04:00
Wing Lian
894cba09f3 fix FSDP save of final model (#329) 2023-07-30 21:46:44 -04:00
Wing Lian
41a4d15d43 update README for updated docker images (#328) 
* update README for updated docker images

* update readme from pr feedback
 2023-07-28 16:50:03 -04:00
Wing Lian
2c37bf6c21 Prune cuda117 (#327) 
* drop cuda117/torch 1.13.1 from support, pin flash attention to v2.0.1, rm torchvision/torchaudio install

* gptq base build not needed. add sm 9.0 support
 2023-07-26 16:27:49 -04:00
Wing Lian
9f69c4d8c1 latest HEAD of accelerate causes 0 loss immediately w FSDP (#321) 2023-07-24 11:23:56 -04:00
Wing Lian
3d4984b9a5 update prompts for open orca to match the paper (#317) 
fix the test for the updated system tokenizer
 2023-07-22 13:49:11 -04:00
Wing Lian
ff7f18d1ed disable gh cache for first step of docker builds too 2023-07-22 11:46:37 -04:00
Wing Lian
cf62cfd661 add runpod envs to .bashrc, fix bnb env (#316) 
* hopper support for base dockerfile, add runpod envs to .bashrc

* set BNB_CUDA_VERSION env for latest bnb

* don't support hopper yet w 118
 2023-07-22 10:09:38 -04:00
Wing Lian
c5df969262 don't use the gha cache w docker 2023-07-22 08:46:21 -04:00
Wing Lian
40a53ff181 Merge pull request #307 from OpenAccess-AI-Collective/xgen-user-sharegpt-tokens 
better handling since xgen tokenizer breaks with convert_tokens_to_ids
 2023-07-22 04:10:38 -04:00
Wing Lian
dcdec44347 Merge pull request #306 from ethanhs/xgen 
Add XGen info to README and example config
 2023-07-22 04:10:18 -04:00
Wing Lian
3ffb018a4c Merge pull request #313 from OpenAccess-AI-Collective/tokenizer-llama2-embeddings 
don't resize embeddings to multiples of 32x by default
 2023-07-22 04:09:59 -04:00
Wing Lian
a94f2eecb1 Merge pull request #299 from OpenAccess-AI-Collective/flash-attention-2 
Flash attention 2
 2023-07-22 04:07:48 -04:00
Wing Lian
1066751358 don't resize embeddings to multiples of 32x by default 2023-07-22 01:52:38 -04:00
Wing Lian
1b63bf13bc Merge pull request #308 from OpenAccess-AI-Collective/apache2-license 
add apache 2.0 license
 2023-07-21 09:50:14 -04:00
Wing Lian
5cce2a42ff add apache 2.0 license 2023-07-21 09:49:29 -04:00
Wing Lian
2a428e8014 better handling since xgen tokenizer breaks with convert_tokens_to_ids 2023-07-21 09:24:11 -04:00
Wing Lian
cdf85fdbd5 pin flash attention 2 to the fix for backwards pass 2023-07-21 08:18:53 -04:00
Wing Lian
9b790d359b flash attention 2 2023-07-21 08:17:46 -04:00
Ethan Smith
38811434e6 Add XGen info to README and example config 2023-07-21 00:44:50 -07:00
NanoCode012
06c61d6f13 Merge pull request #304 from OpenAccess-AI-Collective/NanoCode012-patch-1 
Fix(readme): Improve wording for push model
 2023-07-21 13:39:45 +09:00
Wing Lian
262dc29df2 Merge pull request #300 from OpenAccess-AI-Collective/pytorch-201 
Pytorch 2.0.1
 2023-07-21 00:28:38 -04:00
NanoCode012
165907fddb Fix(readme): Improve wording for push model 2023-07-21 11:28:35 +09:00
Wing Lian
a032c9f452 fix sdp attention to use the flash/mem-efficient context manaager 2023-07-20 01:05:48 -04:00
Wing Lian
b06d3e3645 explicitly pin flash attention 1 to v1.0.9 2023-07-20 01:02:08 -04:00
Wing Lian
c58034d48c use pytorch 2.0.1 2023-07-20 00:47:13 -04:00
NanoCode012
28fd429bcf Merge pull request #293 from NanoCode012/fix/tokenize-speed 
Fix(tokenizing): Use multi-core
 2023-07-19 11:02:04 +09:00
NanoCode012
45ac7c4f88 feat: use multi-core 2023-07-19 10:16:54 +09:00
Wing Lian
edd6980dd9 Merge pull request #289 from OpenAccess-AI-Collective/hf_transfer 
add hf_transfer to requirements for faster hf upload
 2023-07-17 15:08:06 -04:00
Wing Lian
dc6d25124d Merge pull request #288 from OpenAccess-AI-Collective/NanoCode012-patch-1 
fix(readme): remove accelerate config
 2023-07-17 14:46:43 -04:00
Wing Lian
6dd2e7d671 add hf_transfer to requirements for faster hf upload 2023-07-17 14:44:48 -04:00
NanoCode012
b64f411849 fix(readme): remove accelerate config 2023-07-18 01:31:02 +09:00
Wing Lian
03a59c1ed4 Merge pull request #287 from OpenAccess-AI-Collective/dataclass-fix 
fix axolotl training args dataclass annotation
 2023-07-17 06:09:23 -04:00
Wing Lian
ebaec3c406 fix axolotl training args dataclass annotation 2023-07-17 04:57:02 -04:00
Wing Lian
73e70e3996 Merge pull request #286 from OpenAccess-AI-Collective/logging-docker-fixes 
misc fixes
 2023-07-17 04:26:39 -04:00
Wing Lian
d75adb9835 misc fixes 2023-07-17 03:00:27 -04:00
Wing Lian
02224668c3 Merge pull request #283 from OpenAccess-AI-Collective/docker-git-fetch 
git fetch fix for docker
 2023-07-17 02:17:00 -04:00
Wing Lian
f162f3c7cc set transformers cache env var in docker image 2023-07-16 23:03:54 -04:00
Wing Lian
eca3531329 git fetch fix for docker 2023-07-16 22:25:05 -04:00
Wing Lian
6f16c4569d Merge pull request #276 from theobjectivedad/logging_enhancement 
Logging update: added PID and formatting
 2023-07-16 17:04:52 -04:00
Wing Lian
0bd09c077d Merge pull request #280 from teknium1/main 
Update requirements.txt
 2023-07-16 16:08:58 -04:00
Wing Lian
469c08c9ba Merge pull request #279 from NanoCode012/feat/multi-gpu-readme 
Feat(readme): improve docs on multi-gpu
 2023-07-16 16:08:37 -04:00
Wing Lian
334af625d0 Merge pull request #277 from cg123/dataset-name 
Allow non-default dataset configurations
 2023-07-16 16:08:15 -04:00
Teknium
273b3a3aa7 Update requirements.txt 
Require latest git accelerate to fix saving checkpoint issue
 2023-07-16 10:24:24 -07:00
Charles Goddard
3cdd8e4122 Add dataset name to all yaml options in README 2023-07-15 13:17:37 -07:00
NanoCode012
cf5ae6b649 Feat(readme): improve docs on multi-gpu 2023-07-16 01:07:27 +09:00
theobjectivedad
b1f4f7a34d Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var 2023-07-15 12:29:35 +00:00
The Objective Dad
83237b8445 Merge branch 'OpenAccess-AI-Collective:main' into logging_enhancement 2023-07-15 06:16:04 -05:00
Charles Goddard
46032a1a1f Fix formatting mistake 2023-07-14 20:57:27 -07:00