tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Commit Graph

  • 328c3bce96 Merge pull request #149 from OpenAccess-AI-Collective/docker-clone-axolotl Wing Lian 2023-06-02 15:15:30 -04:00
  • 5cd2126439 shallow clone Wing Lian 2023-06-02 14:54:28 -04:00
  • 12620f3089 clone in docker Wing Lian 2023-06-02 14:52:50 -04:00
  • 4ab0c8b201 Merge pull request #148 from OpenAccess-AI-Collective/fix-device-load Wing Lian 2023-06-02 14:37:17 -04:00
  • 74ebbf4371 fix device map Wing Lian 2023-06-02 14:29:08 -04:00
  • 76a70fd739 Merge pull request #147 from OpenAccess-AI-Collective/winglian-rocker-images Wing Lian 2023-06-02 14:10:40 -04:00
  • 618816d4df Update README.md for correct image tags Wing Lian 2023-06-02 14:10:23 -04:00
  • 91992cb8f5 Merge pull request #146 from FarisHijazi/main Wing Lian 2023-06-02 13:58:23 -04:00
  • 84169d15b3 added docker-compose file FarisHijazi 2023-06-02 18:17:43 +03:00
  • ecfe8d0a1a Merge pull request #142 from NanoCode012/feat/custom-prompt-readme Wing Lian 2023-06-02 07:21:04 -04:00
  • eee44a3b47 Merge pull request #141 from NanoCode012/feat/lambdalabs-readme Wing Lian 2023-06-02 07:20:12 -04:00
  • 078a43eef8 Remove redundant instruction NanoCode012 2023-06-02 12:30:11 +09:00
  • 33e1890086 Add pygmalion NanoCode012 2023-06-02 12:27:51 +09:00
  • 1c38253692 Add other prompt_strategies NanoCode012 2023-06-02 12:24:44 +09:00
  • 496b83f778 Add short instruction for custom prompts NanoCode012 2023-06-02 12:16:20 +09:00
  • ff68a95781 Add lambdalabs instruction NanoCode012 2023-06-02 12:09:40 +09:00
  • 6fcb73faaa more gpt-neox long ctx fixes exp-expand-len Wing Lian 2023-06-01 08:20:08 -04:00
  • fb3d40f197 falcon + qlora + xformer mbs 40 gas 2 on A6000 Utensil 2023-06-01 18:29:20 +08:00
  • a32cc1d021 fix bettertransformers save, force it to skip after saving correctly in callback Wing Lian 2023-06-01 00:33:13 -04:00
  • 86bd9fcff4 more tweaks to do pre-training with bettertransformers Wing Lian 2023-05-31 21:59:15 -04:00
  • 288fd62431 Merge pull request #135 from NanoCode012/fix/grad-accu-readme NanoCode012 2023-06-01 06:33:05 +09:00
  • 3c71c8debe Update doc for grad_accu and add validation tests for batch size NanoCode012 2023-06-01 06:13:47 +09:00
  • ed7531abb8 experimental expansion of ctx len Wing Lian 2023-05-31 16:51:19 -04:00
  • bdb547b830 add validation/warning for bettertransformers and torch version Wing Lian 2023-05-28 08:56:08 -04:00
  • 8a37b43678 use pythia-12b, neox-20b is flaky Wing Lian 2023-05-27 19:37:24 -04:00
  • 28acebac36 add flash attn context for efficient training and attempt setting model to train mode: Wing Lian 2023-05-27 18:12:12 -04:00
  • adea682316 add support for opimum bettertransformers Wing Lian 2023-05-27 17:57:29 -04:00
  • a6f5e5eaec Merge pull request #134 from OpenAccess-AI-Collective/gas-batch-fix Wing Lian 2023-05-31 14:24:48 -04:00
  • 5a631b305b fix batch size calculation Wing Lian 2023-05-31 14:11:32 -04:00
  • f94dd626f0 Merge pull request #130 from OpenAccess-AI-Collective/gas Wing Lian 2023-05-31 13:03:51 -04:00
  • 5079753b7a Merge pull request #131 from OpenAccess-AI-Collective/fix-packing-mask Wing Lian 2023-05-31 13:03:37 -04:00
  • 0136f510f2 don't worry about duplicate code here Wing Lian 2023-05-31 12:05:43 -04:00
  • 72bf8aafb6 Create config-7b-qlora.yml Utensil 2023-06-01 00:00:37 +08:00
  • 8afb0fbaba Axolotl supports falcon + qlora Utensil 2023-05-31 23:58:40 +08:00
  • 9b8585dc70 fix packing so that concatenated sequences reset the attention Wing Lian 2023-05-31 11:38:52 -04:00
  • 8eb5811d4e Merge pull request #129 from OpenAccess-AI-Collective/builder-badge Wing Lian 2023-05-31 10:37:59 -04:00
  • e0011fdf55 Fix base builder, missing tags Wing Lian 2023-05-31 09:52:03 -04:00
  • 6e9e98720e Merge pull request #127 from OpenAccess-AI-Collective/py310-docker-runpod Wing Lian 2023-05-31 09:39:42 -04:00
  • c2a0792680 swap batch size for gradient accumulation steps to decouple from num gpu Wing Lian 2023-05-31 09:38:12 -04:00
  • b267d24a2b add badge info to readme Wing Lian 2023-05-31 09:28:44 -04:00
  • 5c3f5db38b Add files via upload Wing Lian 2023-05-31 09:22:54 -04:00
  • e3d03745ba add py310 support from base image Wing Lian 2023-05-31 09:07:28 -04:00
  • fac46002d4 Merge pull request #119 from NanoCode012/feat/update-inference NanoCode012 2023-05-31 14:09:18 +09:00
  • 33d40179ba Increase max_new_tokens NanoCode012 2023-05-31 14:04:49 +09:00
  • dcb03d6da4 Merge pull request #114 from OpenAccess-AI-Collective/accelerate-dep Wing Lian 2023-05-31 00:47:17 -04:00
  • 0e4be625ae Merge pull request #118 from NanoCode012/feat/torch-readme NanoCode012 2023-05-31 13:29:41 +09:00
  • bdc4bd7d4e Update README.md NanoCode012 2023-05-31 13:24:28 +09:00
  • 2d0ba3b818 Merge pull request #124 from OpenAccess-AI-Collective/xformers-fix Wing Lian 2023-05-31 00:11:40 -04:00
  • c7021e191f Merge pull request #120 from OpenAccess-AI-Collective/model-from-path Wing Lian 2023-05-31 00:08:38 -04:00
  • c56818b119 don't worry about dupes Wing Lian 2023-05-31 00:06:47 -04:00
  • 2675fb756e update readme for SDP Wing Lian 2023-05-31 00:02:29 -04:00
  • 1076bcbbca Update src/axolotl/monkeypatch/llama_attn_hijack_xformers.py Wing Lian 2023-05-31 00:00:19 -04:00
  • 2daa6835f0 Update src/axolotl/monkeypatch/llama_attn_hijack_xformers.py Wing Lian 2023-05-30 23:59:05 -04:00
  • e3c494ca7b remove unused import and update readme Wing Lian 2023-05-30 23:55:45 -04:00
  • ad0ea6aaab black formatting Wing Lian 2023-05-30 23:36:01 -04:00
  • 876edd83d0 Merge pull request #123 from OpenAccess-AI-Collective/bas-batch Wing Lian 2023-05-30 23:45:29 -04:00
  • 6cb2310592 copy xformers attn from ooba since we removed dep on alpaca_lora_4bit Wing Lian 2023-05-30 23:34:36 -04:00
  • 6fa40bf8ad black formatting Wing Lian 2023-05-30 23:33:37 -04:00
  • 3aad5f3b3e add support for gradient accumulation steps Wing Lian 2023-05-30 23:24:37 -04:00
  • 39a208c2bc fix up tokenizer config, isort fix Wing Lian 2023-05-30 23:00:02 -04:00
  • 2520ecd6df split up llama model loading so config can be loaded from base config and models can be loaded from a path Wing Lian 2023-05-30 22:32:44 -04:00
  • c5b0af1a7e define python version (3.10) explicitly as string in yaml Wing Lian 2023-05-30 22:23:35 -04:00
  • 988aeb9c34 Feat: Swap to GenerationConfig NanoCode012 2023-05-31 10:48:19 +09:00
  • cf61f14bff FIx(readme): Fix torch missing from readme NanoCode012 2023-05-31 10:28:49 +09:00
  • 0abcd71a85 Merge pull request #115 from OpenAccess-AI-Collective/docker-version-fixes Wing Lian 2023-05-30 18:11:26 -04:00
  • c43c5c84ff py310, fix cuda arg in deepspeed Wing Lian 2023-05-30 18:02:34 -04:00
  • 36ec6e1a0e Add accelerate dep Wing Lian 2023-05-30 16:36:13 -04:00
  • 13b80937f9 add release draft template for gh v0.2.0 Wing Lian 2023-05-30 15:10:19 -04:00
  • bbc5bc5791 Merge pull request #108 from OpenAccess-AI-Collective/docker-gptq Wing Lian 2023-05-30 15:07:04 -04:00
  • 4df9da74e3 Merge pull request #105 from viktoriussuwandi/viktoriussuwandi-patch Wing Lian 2023-05-30 15:05:23 -04:00
  • 2531ea24c1 Merge pull request #106 from fearnworks/qlora-openllama-3b-example Wing Lian 2023-05-30 15:05:05 -04:00
  • 01a75fd027 Merge pull request #98 from NanoCode012/feat/pre-commit Wing Lian 2023-05-30 14:57:15 -04:00
  • b81c97ff76 Fix pre-commit for rebased files NanoCode012 2023-05-31 03:01:38 +09:00
  • 594e72b6e8 Fix incorrect rebase NanoCode012 2023-05-31 02:58:50 +09:00
  • 25eeeeba0b Fix sharegpt prompt NanoCode012 2023-05-31 00:38:08 +09:00
  • cfcc549f6b fix relative path for fixtures Wing Lian 2023-05-30 10:38:20 -04:00
  • a1f9850b91 Fix security issue or ignore false positives NanoCode012 2023-05-29 22:26:26 +09:00
  • 83d29209f7 Add bandit NanoCode012 2023-05-29 22:25:59 +09:00
  • d011422200 Add isort NanoCode012 2023-05-29 21:53:29 +09:00
  • b1cc54b14a Update pip install to also setup tests NanoCode012 2023-05-29 21:49:39 +09:00
  • c17dae6d07 Update src/axolotl/prompt_strategies/alpaca_instruct.py NanoCode012 2023-05-29 21:46:05 +09:00
  • 37293dce07 Apply isort then black NanoCode012 2023-05-29 18:48:58 +09:00
  • 96e8378692 Delete extract_lora.py NanoCode012 2023-05-29 18:14:33 +09:00
  • e9650d3ae4 Fix mypy typing NanoCode012 2023-05-29 18:13:39 +09:00
  • f1232b35ba Update mypy dependencies NanoCode012 2023-05-29 18:04:17 +09:00
  • 741a3f2edc Add mypy NanoCode012 2023-05-29 17:35:51 +09:00
  • 0dd35c74af Ignore unsupported-binary-operation NanoCode012 2023-05-29 16:54:19 +09:00
  • db288e9b13 Set python version NanoCode012 2023-05-29 15:51:32 +09:00
  • be22551435 Fix unsupported operand type(s) for | NanoCode012 2023-05-29 15:33:40 +09:00
  • b832a0ac62 Black formatting NanoCode012 2023-05-29 15:30:28 +09:00
  • afb31e13a3 Add badge and update contribution section NanoCode012 2023-05-29 15:24:54 +09:00
  • 1bf1f59a41 Move black to dev requirements NanoCode012 2023-05-29 15:24:40 +09:00
  • 8e46c0fb0d Refactor duplicate code between Prompter and Pygmalion NanoCode012 2023-05-29 15:08:26 +09:00
  • 1f3c3f5ea0 Lint validation NanoCode012 2023-05-29 14:29:19 +09:00
  • 0e952889dc Lint test_dict NanoCode012 2023-05-29 14:28:38 +09:00
  • 9c6750a075 Lint wandb NanoCode012 2023-05-29 14:27:08 +09:00
  • c2dbf2c526 Lint validation NanoCode012 2023-05-29 14:26:43 +09:00
  • e6b57decbd Lint tokenization NanoCode012 2023-05-29 14:26:12 +09:00
  • fe1f4c4e7d Lint schedulers NanoCode012 2023-05-29 14:25:15 +09:00
  • dae14e5951 Ignore too-many-instance-attributes NanoCode012 2023-05-29 14:23:23 +09:00