tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

consolidate as peft_model_dir

Browse Source
This commit is contained in:
Wing Lian
2023-09-19 19:02:14 -04:00
parent ba85308720
commit 203369411e
33 changed files with 50 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@@ -96,7 +96,7 @@ accelerate launch -m axolotl.cli.train examples/openllama-3b/lora.yml
# inference
accelerate launch -m axolotl.cli.inference examples/openllama-3b/lora.yml \
--lora_model_dir="./lora-out"
--peft_model_dir="./lora-out"
```
## Installation
@@ -531,7 +531,7 @@ total_num_tokens:
adapter: lora
# If you already have a lora model trained that you want to load, put that here.
# This means after training, if you want to test the model, you should set this to the value of `lora_out_dir`.
lora_model_dir:
peft_model_dir:
# LoRA hyperparameters
# For more details about the following options, see:
@@ -869,7 +869,7 @@ Pass the appropriate flag to the train command:
- Pretrained LORA:
```bash
python -m axolotl.cli.inference examples/your_config.yml --lora_model_dir="./lora-output-dir"
python -m axolotl.cli.inference examples/your_config.yml --peft_model_dir="./lora-output-dir"
```
- Full weights finetune:
```bash
@@ -890,7 +890,7 @@ Please use `--sample_packing False` if you have it on and receive the error simi
Add below flag to train command above
```bash
python3 -m axolotl.cli.merge_lora examples/your_config.yml --lora_model_dir="./completed-model" --load_in_8bit=False --load_in_4bit=False
python3 -m axolotl.cli.merge_lora examples/your_config.yml --peft_model_dir="./completed-model" --load_in_8bit=False --load_in_4bit=False
```
If you run out of CUDA memory, you can try to merge in system RAM with