qlora merge and load requires that base model isn't loaded in 4 or 8 bit

This commit is contained in:
Wing Lian
2023-05-25 22:39:13 -04:00
parent 34c99f9812
commit 3f6017db9e

View File

@@ -363,12 +363,18 @@ Pass the appropriate flag to the train command:
### Merge LORA to base ### Merge LORA to base
Add below flag to train command above Add below flag to train command above (and using LoRA)
```bash ```bash
--merge_lora --lora_model_dir="./completed-model" --merge_lora --lora_model_dir="./completed-model"
``` ```
Add below flag to train command above (and using QLoRA)
```bash
--merge_lora --lora_model_dir="./completed-model" --load_in_8bit False --load_in_4bit False
```
## Common Errors 🧰 ## Common Errors 🧰
> Cuda out of memory > Cuda out of memory