tocmo0nlord/axolotl
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

feat: remove need to add load_in* during merge (#1017)

Browse Source
This commit is contained in:
NanoCode012
2023-12-29 18:15:30 +09:00
committed by GitHub
parent dec66d7c53
commit f6ecf14dd4
2 changed files with 11 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -1027,7 +1027,7 @@ Please use `--sample_packing False` if you have it on and receive the error simi
Add below flag to train command above Add below flag to train command above
```bash ```bash
python3 -m axolotl.cli.merge_lora examples/your_config.yml --lora_model_dir="./completed-model" --load_in_8bit=False --load_in_4bit=False python3 -m axolotl.cli.merge_lora examples/your_config.yml --lora_model_dir="./completed-model"
``` ```
If you run out of CUDA memory, you can try to merge in system RAM with If you run out of CUDA memory, you can try to merge in system RAM with

10
src/axolotl/cli/merge_lora.py
View File

@@ -18,7 +18,15 @@ def do_cli(config: Path = Path("examples/"), **kwargs):
return_remaining_strings=True return_remaining_strings=True
) )
parsed_cli_args.merge_lora = True parsed_cli_args.merge_lora = True
parsed_cfg = load_cfg(config, merge_lora=True, **kwargs)
parsed_cfg = load_cfg(
config,
merge_lora=True,
load_in_8bit=False,
load_in_4bit=False,
flash_attention=False,
**kwargs
)
do_merge_lora(cfg=parsed_cfg, cli_args=parsed_cli_args) do_merge_lora(cfg=parsed_cfg, cli_args=parsed_cli_args)