qlora merge and load requires that base model isn't loaded in 4 or 8 bit
This commit is contained in:
@@ -363,12 +363,18 @@ Pass the appropriate flag to the train command:
|
||||
|
||||
### Merge LORA to base
|
||||
|
||||
Add below flag to train command above
|
||||
Add below flag to train command above (and using LoRA)
|
||||
|
||||
```bash
|
||||
--merge_lora --lora_model_dir="./completed-model"
|
||||
```
|
||||
|
||||
Add below flag to train command above (and using QLoRA)
|
||||
|
||||
```bash
|
||||
--merge_lora --lora_model_dir="./completed-model" --load_in_8bit False --load_in_4bit False
|
||||
```
|
||||
|
||||
## Common Errors 🧰
|
||||
|
||||
> Cuda out of memory
|
||||
|
||||
Reference in New Issue
Block a user