From 2b222de5b6f10a169c3eb8ecb0fa23d0093a9ac6 Mon Sep 17 00:00:00 2001 From: NanoCode012 Date: Thu, 8 Jun 2023 22:48:26 +0900 Subject: [PATCH 1/2] Update peft and gptq instruction --- README.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index db884ec6b..b480c798e 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,7 @@ accelerate launch scripts/finetune.py examples/lora-openllama-3b/config.yml \ docker run --gpus '"all"' --rm -it winglian/axolotl:main-py3.9-cu118-2.0.0 ``` - `winglian/axolotl-runpod:main-py3.9-cu118-2.0.0`: for runpod + - `winglian/axolotl-runpod:main-py3.9-cu118-2.0.0-gptq`: for gptq - `winglian/axolotl:dev`: dev branch (not usually up to date) Or run on the current files for development: @@ -67,9 +68,19 @@ accelerate launch scripts/finetune.py examples/lora-openllama-3b/config.yml \ 2. Install pytorch stable https://pytorch.org/get-started/locally/ 3. Install python dependencies with ONE of the following: - - `pip3 install -e .` (recommended, supports QLoRA, no gptq/int4 support) - - `pip3 install -e .[gptq]` (next best if you don't need QLoRA, but want to use gptq) - - `pip3 install -e .[gptq_triton]` + - Recommended, supports QLoRA, NO gptq/int4 support + ```bash + pip3 install -U git+https://github.com/huggingface/peft.git + pip3 install -e . + ``` + - gptq/int4 support, NO QLoRA + ```bash + pip3 install -e .[gptq] + ``` + - same as above but not recommended + ```bash + pip3 install -e .[gptq_triton] + ``` - LambdaLabs
From cfff94b123d6c5161fd605cf1ad22844d9cf27b0 Mon Sep 17 00:00:00 2001 From: NanoCode012 Date: Thu, 8 Jun 2023 22:50:20 +0900 Subject: [PATCH 2/2] Add peft install for quickstart --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index b480c798e..68e3f76f5 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ git clone https://github.com/OpenAccess-AI-Collective/axolotl pip3 install -e . +pip3 install -U git+https://github.com/huggingface/peft.git accelerate config @@ -70,8 +71,8 @@ accelerate launch scripts/finetune.py examples/lora-openllama-3b/config.yml \ 3. Install python dependencies with ONE of the following: - Recommended, supports QLoRA, NO gptq/int4 support ```bash - pip3 install -U git+https://github.com/huggingface/peft.git pip3 install -e . + pip3 install -U git+https://github.com/huggingface/peft.git ``` - gptq/int4 support, NO QLoRA ```bash