qlora merge and load requires that base model isn't loaded in 4 or 8 bit
This commit is contained in:
@@ -363,12 +363,18 @@ Pass the appropriate flag to the train command:
|
|||||||
|
|
||||||
### Merge LORA to base
|
### Merge LORA to base
|
||||||
|
|
||||||
Add below flag to train command above
|
Add below flag to train command above (and using LoRA)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
--merge_lora --lora_model_dir="./completed-model"
|
--merge_lora --lora_model_dir="./completed-model"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Add below flag to train command above (and using QLoRA)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--merge_lora --lora_model_dir="./completed-model" --load_in_8bit False --load_in_4bit False
|
||||||
|
```
|
||||||
|
|
||||||
## Common Errors 🧰
|
## Common Errors 🧰
|
||||||
|
|
||||||
> Cuda out of memory
|
> Cuda out of memory
|
||||||
|
|||||||
Reference in New Issue
Block a user